By Dr Joshua Scarpino

The Gap Between Certified and Governed

By Dr. Joshua Scarpino, D.Sc.

An AI governance officer recently posted a candid admission on r/cybersecurity. He had led his employer through ISO/IEC 42001 certification and passed with a single audit finding. Instead of celebrating, he questioned whether AI governance is a legitimate discipline at all. His auditors never challenged him. He built the program, cleared the audit, and walked away doubting the entire field.

I wrote about his experience on LinkedIn, and the response was substantial: more than 21,000 impressions and 80 comments from CISOs, auditors, attorneys, and governance leads across the industry in a relatively short period of time. The discussion sharpened the original argument in ways worth capturing, because his skepticism points at a real structural problem. What he encountered was attestation, and attestation is not the discipline he thought he was practicing.

Link to post –> https://www.linkedin.com/posts/joshuascarpino_an-ai-governance-officer-expressed-disappointment-activity-7487480665295273984-eQ20?utm_source=share&utm_medium=member_desktop&rcm=ACoAAAo5QskBLWM0ENIKlcURT7TmvMTguGfBq74

Two Different Disciplines Wearing One Name

A certificate reflects a program at a specific moment. ISO/IEC 42001:2023 is the first certifiable international standard for an Artificial Intelligence Management System (AIMS), and it holds genuine value as a structural foundation [1, 2]. The certification model behind it, however, operates on a three-year cycle with annual surveillance audits [3, 4]. The systems it governs do not operate on any cycle at all. Models retrain, permissions expand, vendors ship new capabilities, and use cases multiply between every audit touchpoint.

Muhammad Ali, a banking CISO, stated the distinction cleanly in the discussion: passing an audit tells you that controls exist; governance tells you whether those controls still work as the AI system, data, and business context change. Chris Thornberry extended the point in a direction I keep returning to. Governance is becoming less about proving you were compliant at a point in time and more about proving your decisions were reasonable based on what you knew when you made them. That is a higher standard of assurance, and for systems that change daily, it is the right one.

The Evidence Behind the Drift Problem

The claim that AI risk profiles shift between audits is not intuition; it is measured. A study published in Nature Scientific Reports tested 128 model-dataset pairs across weather, healthcare, transportation, and finance, and found temporal degradation in 91% of cases, including failure patterns that data drift monitoring alone could not predict [5, 6]. A model that performed well on the day evidence was collected can degrade quietly for months while the certificate stays current.

The consequences are visible in the incident data. Stanford’s 2025 AI Index recorded 233 AI incidents in 2024, a 56% increase over the prior year, and noted that reliance on public media reports means the true number is likely higher [7, 8]. IBM’s 2025 Cost of a Data Breach report adds the governance dimension: 13% of organizations reported breaches of AI models or applications, 97% of those breached lacked proper AI access controls, and 63% of breached organizations had no AI governance policy in place or were still developing one [9, 10]. Certificates were never designed to detect any of this. Continuous governance was.

The Auditor Side of the Gap

The officer’s program was only half of his story. The other half was an audit that never pushed back. Alena Elmer named it precisely in the thread: if the auditor did not challenge anything, the question is whether the auditor understood the AI risk well enough to know what to challenge. That is a competency gap on the audit side, not just a scope problem.

Researchers Ellen Goodman and Julia Tréhu warned about this dynamic in their German Marshall Fund paper on “audit-washing”: an audit ecosystem without defined standards, auditor competence requirements, and real independence can legitimize practices rather than scrutinize them [11, 12]. Eric Coomer’s comment in the discussion described the mechanism. Attestation has a presentation layer; an organization assembles evidence, and someone reviews what was assembled. Any process with a presentation layer can only verify what was presented. Closing that gap requires two moves: certification criteria written as binary, auditable requirements so outcomes depend less on individual judgment, which is the approach we take at ForHumanity, and dedicated competency development for the auditors themselves. Until both catch up, certificates will keep telling us less than they should.

The Legal Standard Is Already Continuous

Karthik Jayaraman brought the framing that elevates this from professional debate to legal exposure: duty of care runs continuously, and pointing to a point-in-time attestation after an incident will not satisfy regulatory scrutiny. The regulatory architecture agrees with him. Article 72 of the EU AI Act requires providers of high-risk AI systems to establish post-market monitoring that actively and systematically collects and analyzes performance data across the system’s lifetime, expressly to evaluate continuous compliance [13, 14].

Cybersecurity law reached this conclusion years ago. Ohio’s Data Protection Act grants an affirmative defense only to organizations that create, maintain, and comply with a written cybersecurity program conforming to a recognized framework [15, 16]. The operative word is maintain. The defense is earned through operation, not through a certificate on the wall. A regulator examining an AI incident will ask what the organization knew and did in the months between audits, and the only record that answers that question is operational: the living inventory, the drift detection, the decisions made when monitoring surfaced a change.

What Governed Actually Looks Like

Meaningful governance starts with a comprehensive inventory of every AI system in the environment, because you cannot assess what you have not enumerated. It establishes foundational controls before harm occurs rather than after. It builds feedback loops that reveal drift in real time. It integrates cybersecurity, AI, privacy, and ethics into one unified lens rather than allowing each function to attest to its own segment while risk accumulates in the gaps between them. Unified governance gives you the complete picture, and when you have the complete picture you can see when pieces are missing. This is the operating model we formalized in the ARISE Framework™, which maps to ISO and NIST rather than replacing them.

Edna O. asked the question the entire discussion leads to: how do we measure whether an AI governance program is actually effective six months after certification? Most organizations have not defined what to measure. I would start with what is observable between audits. Confirm the AI system inventory is still complete six months on. Measure how quickly drift gets detected and closed once it begins. Verify that monitoring findings changed a deployment decision or a permission rather than accumulating in a report. Confirm that someone in the accountability structure has made a real decision under it, because an escalation path that has never carried an escalation is a diagram, not a control. None of this appears on the certificate, and all of it is measurable today.

The Certificate Is the Starting Line

The officer who posted his doubts deserves a direct answer. The discipline is legitimate; the assurance model he experienced is incomplete. Organizations should pursue ISO/IEC 42001, because the work of building an AIMS produces real structure. They must not confuse that milestone with the outcome. The certificate marks the start of the work, and the organizations that fund what happens between audits, rather than only the audits themselves, are the ones that will hold up when the systems, the threats, and the regulators all keep moving.

References

1. International Organization for Standardization. ISO/IEC 42001:2023, AI Management Systems. https://www.iso.org/standard/42001
2. BSI Group. ISO/IEC 42001 AI Management System. https://www.bsigroup.com/en-US/products-and-services/standards/iso-42001-ai-management-system/
3. Schellman. What to Expect in the ISO 42001 Certification Process. https://www.schellman.com/blog/iso-certifications/iso-42001-certification-processs
4. Cloud Security Alliance. What to Expect in the ISO 42001 Certification Process. https://cloudsecurityalliance.org/articles/what-to-expect-in-the-iso-42001-certification-process
5. Vela, D., et al. “Temporal Quality Degradation in AI Models.” Scientific Reports 12, 11654 (2022). https://www.nature.com/articles/s41598-022-15245-z
6. NannyML. “91% of ML Models Degrade in Time.” https://www.nannyml.com/blog/91-of-ml-perfomance-degrade-in-time
7. Stanford HAI. The 2025 AI Index Report, Responsible AI chapter. https://hai.stanford.edu/ai-index/2025-ai-index-report/responsible-ai
8. Kiteworks. “AI Data Privacy Risks Surge 56%: Critical Findings from Stanford’s 2025 AI Index Report.” https://www.kiteworks.com/cybersecurity-risk-management/ai-data-privacy-risks-stanford-index-report-2025/
9. IBM Newsroom. “IBM Report: 13% of Organizations Reported Breaches of AI Models or Applications, 97% of Which Reported Lacking Proper AI Access Controls.” July 30, 2025. https://newsroom.ibm.com/2025-07-30-ibm-report-13-of-organizations-reported-breaches-of-ai-models-or-applications,-97-of-which-reported-lacking-proper-ai-access-controls
10. Jones Walker LLP. “The AI Oversight Gap: IBM’s 2025 Data Breach Report Reveals Hidden Costs of Ungoverned AI.” https://www.joneswalker.com/en/insights/blogs/ai-law-blog/the-ai-oversight-gap-ibms-2025-data-breach-report-reveals-hidden-costs-of-ungov.html
11. Goodman, E. P., and Tréhu, J. “AI Audit-Washing and Accountability.” German Marshall Fund of the United States, November 2022. https://www.gmfus.org/news/ai-audit-washing-and-accountability
12. Goodman, E. P., and Tréhu, J. “Algorithmic Auditing” (full policy paper PDF). https://www.gmfus.org/sites/default/files/2022-11/Goodman%20%26%20Trehu%20-%20Algorithmic%20Auditing%20-%20paper.pdf
13. EU Artificial Intelligence Act, Article 72: Post-Market Monitoring by Providers. https://artificialintelligenceact.eu/article/72/
14. EU AI Act (euaiact.com), Article 72. https://www.euaiact.com/article/72
15. Ohio Revised Code § 1354.02, Safe Harbor Requirements. https://codes.ohio.gov/ohio-revised-code/section-1354.02
16. Jones Day. “Ohio Adopts Safe Harbor for Businesses Involved in Data Breach.” October 2018. https://www.jonesday.com/en/insights/2018/10/ohio-adopts-safe-harbor-for-businesses-involved-in