In January, a small Florida-based company was found in contempt of court and fined tens of thousands of dollars, after which it promptly dissolved. For Cyber Ninjas, this was only the latest chapter in a tumultuous year since it was contracted to evaluate the 2020 election results in Maricopa County, Arizona. Its review made national news for various stumbles — but these missteps can prove instructive for recognizing future sham reviews, especially when contrasted with other, more credible efforts.
Ten days before Cyber Ninjas released its final report, the American Statistical Association’s Census Quality Indicators Task Force released its quality evaluation of the 2020 Census to comparatively little fanfare. While both the census task force and Cyber Ninjas purported to provide independent oversight of a technical process crucial to a functioning democracy, the former consisted of experts conducting a fair, informed, and transparent audit, while the latter lacked each of these necessary components of credibility. As the next election cycle looms and politicians around the country continue to pursue partisan election reviews, the differences between these two efforts can shed valuable light on how credible audits can and should work. Let’s look at a few of the biggest differences.
Expertise
The U.S. Government Accountability Office requires that auditors possess technical expertise that allows them to accurately identify problems — or a lack thereof. This subject matter expertise ensures any review by auditors is methodologically sound and credible.
Partisan Arizona Review
Cyber Ninjas had no background in election reviews — with not a single mention of auditing on its website prior to the Arizona review — but was chosen over a Boston-based firm that has conducted 200 election audits. Predictably, its report was riddled with errors. Many of Cyber Ninjas’ policy recommendations were already in place. For instance, it recommended that “voting machines… always have a paper backup of all ballots,” even though they already do in every county in Arizona. Implying that policies like these do not exist misinforms Arizonan voters and contributes to declining confidence in elections.
Cyber Ninjas also lacked an understanding of probability and how those concepts play out in voter data. That precluded its ability to accurately assess any potential numerical discrepancies in the election results.
Census Task Force
The census task force, on the other hand, was composed of credible experts representing a broad range of relevant backgrounds, from academic researchers to former Census Bureau directors and scientists. Several are accomplished statisticians and demographers, and some have served as expert witnesses in federal litigation. These experts were assembled by the American Statistical Association, a relevant, credible, professional organization with a centuries-long commitment to gathering, classifying, and reporting U.S. statistics.
Objectivity
The GAO also outlines a baseline standard of objectivity for auditors, a criterion that is exceptionally important in a hyper-polarized political environment. If the review appears biased, the public will be less likely to trust the results.
Partisan Arizona Review
Even if Cyber Ninjas could conduct an impartial assessment, its public support for former president Donald Trump and the Big Lie undermined their review’s credibility, ensuring its perception as a partisan effort. (This perception was only deepened by its apparent lack of interest in auditing the election results until Arizona had been called for President Biden.) CEO Doug Logan had made his pro-Trump and pro-conspiracy views known on social media, authored a memo advising U.S. senators to object to certifying the 2020 election, and he worked closely with Sidney Powell and other Trump allies who sought to prevent President Biden from taking office. Money poured into the company’s review from sources with obvious ulterior motives, such as Overstock CEO Patrick Byrne, who infamously berated Trump officials that they weren’t doing enough to attempt to overturn the election results. This clearly partisan activity creates the impression that the review was not conducted in good faith.
Census Task Force
The task force was formed in response to concerns about the 2020 Census’s data quality before the census was completed or the results were known, rather than after it turned out in a way that partisan actors didn’t approve. The task force continued its work despite a change in political administrations mid-stream and without regard to the political outcomes of the count. This helped bolster the sense that it was impartially pursuing its goal to “produce a set of scientifically-sound, publicly available statistical indicators” on the quality and accuracy of the 2020 census.
In a further demonstration of its objectivity, the task force addressed conflicts of interest as they emerged. When President Biden appointed task force co-chair Robert Santos to be director of the Census Bureau, for instance, Santos immediately resigned from the task force, allowing it to maintain its impartiality and conduct its review separately from the Bureau.
Transparency and Security
Transparency is necessary for public trust because it holds external reviewers accountable for their actions and encourages objective results. A transparent review process should also include protocols that ensure data confidentiality and security while clearly telling the public what can and cannot be shared.
Partisan Arizona Review
Despite promising to conduct “the most transparent election audit in American history,” Cyber Ninjas fought to keep its review procedures secret and restricted press access to the process. Ballots were left unattended and spotted near pens with blue ink, exposing them to tampering.
Census Task Force
The Census Task Force struck a finer balance between data confidentiality and process transparency. The Census Bureau selected three task force members with special sworn status to access confidential, internal 2020 census data, which ensured that their review methods complied with strong confidentiality laws. The task force was also transparent about its membership, assessment methodology, and outputs, as well as its evaluation’s limitations.
The task force’s report advocated for increased transparency within the Census Bureau, recommending that the bureau make its quality indicators publicly available. They also encouraged other qualified external researchers to conduct independent analyses.
Addressing Limitations
As outlined in the GAO’s best practices for identifying sources of evidence, credible external reviews must also refrain from casting baseless aspersions, use clear language about uncertain findings, and modify the scope or methodology of an audit in the face of insufficient evidence.
Partisan Arizona Review
When Cyber Ninjas did not have enough evidence to make clear and credible claims, it made unfounded allegations, such as “23,334 ballots impacted,” or implicit accusations, without providing proof of error or bad intent. In its executive summary, the company recommended that its findings be referred to the Arizona attorney general without making any case for criminal charges in the report. When it did include caveats to their sensationalist claims, they were buried in footnotes.
Census Task Force
When the census task force did not have sufficient data, it said so clearly. In its September 2021 report, it concluded that the “indicators released to date by the bureau do not permit a thorough assessment of the 2020 census data quality.” This transparency is not a weakness — it bolstered the report’s credibility.
• • •
Credible election audits are a critical security measure. Outside experts have an important role to play in government oversight, which is why the Census Bureau has consistently welcomed quality evaluations from various expert organizations. But guardrails are needed to ensure nongovernment entities follow existing professional standards and avoid another Cyber Ninjas–type travesty.
Now more than ever, as several state legislatures take steps to codify these charades that waste taxpayer money and undermine public confidence in elections, we must be able to tell the difference between professional oversight that supports good governance and partisan sham reviews.
