The AI governance gap

Most organizations are only focused on AI risk that stops at the firewall. The exposures that keep underwriters up at night are coming from somewhere else entirely.

6 Min Read

AI risk and cyber insurance are converging faster than most organizations are prepared for. As Chief Underwriting Officer at Resilience, I spend a lot of time thinking about how AI is reshaping the loss landscape, and the gap I keep seeing doesn’t start with  the technology. It’s in who’s in the room when organizations make decisions about how they use AI, govern it, and transfer its risk.

CISOs usually ask me about the threat actor side of AI: how do we defend against AI-powered phishing? What does AI-assisted social engineering look like in practice? How do we combat AI enabled vulnerability discovery and exploitation? Those are real and growing concerns, but they’re only half the picture. The conversation that will shapes the holistic view of AI risk management is centered on how your organization is using AI internally, and whether anyone has mapped that use to financial exposure.

The coverage question most policyholders aren’t asking

When an AI system causes a loss, classification determines coverage. And classification is more complicated than most policyholders realize.

AI introduces liability from two directions. The first is familiar: threat actors using AI against your organization to run more convincing phishing campaigns, generate synthetic media for social engineering fraud, and accelerate attacks in ways that strain traditional defenses. Those losses generally implicate first-party coverages. The second direction is less intuitive: your own use of AI creating third-party liability. When a company uses AI in its products or services, incorporates AI into hiring, lending, or healthcare decisions, or generates content through AI without reviewing it before it goes public, it takes on a different category of exposure entirely, one that can touch technology errors and omissions, media liability, employment practices liability, and professional liability depending on how the loss materializes.

Most cyber policies were drafted before AI became ubiquitous. The result is what the insurance industry calls silent AI (a nod to the silent Cyber days): risk that sits inside a policy that neither explicitly covers it nor explicitly excludes it. Policyholders interpret this as ambiguity in policy language diminishing policyholder confident in the coverage intent.. At Resilience, we’re actively working to address this through affirmative coverage language and new definitions — artificial intelligence, data poisoning attack, data training sets — that make the coverage intent clear. The broader market is still catching up, and policyholders are left with concerns over risk transfer breadth and coverage..

Shadow AI is the risk your underwriter already knows about

Your organization’s lack on AI adoption does not equate to risk avoidance. Spoiler alert: Your employees have been using it for a while. They’ve been using ChatGPT, Gemini, Copilot, and a growing list of AI-enabled tools for years to gain efficiency and at times to fill skill gaps. The absence of a sanctioned AI program, including governance, policies, and defined adoption doesn’t prevent that; it just means it’s happening without your knowledge, without training, and without any guardrails on what information is going in.

What goes into a public LLM doesn’t stay within your organization. Employees who paste intellectual property, personally identifiable information, or protected health information into an external AI platform may be creating a privacy exposure while trying to save an hour of work. Underwriters are starting to ask about this directly, and organizations without ananswer are creating uncertainty at exactly the wrong moment in the placement process.

Prohibiting AI use doesn’t work and creates its own risks. Organizations that formally adopt AI, choose the sanctioned tools, set acceptable use policies, and train employees on what to do and not do are in a better-documented risk position than those that look away. Control over which AI systems your employees use, and how, is the prerequisite for everything else in AI governance.

The stakeholders missing from your AI risk conversation

AI governance is no longer exclusively a security function, and the exposure that’s moved furthest outside the CISO’s traditional remit is media liability.

Media liability is a coverage component in most cyber policies, and AI-generated content creates exposure in at least two ways. Generative AI trained on copyrighted material is the subject of substantial ongoing litigation over whether that training constitutes infringement, and the outcome will have downstream implications for any organization using foundational AI models. Separately, organizations that use AI to generate content and don’t review it before publication are taking on defamation and reputational risk that the policy may or may not cover depending on how it’s worded.

Your content and marketing teams are almost certainly using generative AI right now. They’re excited about it, and it’s genuinely useful for what they do. But they’re not thinking about what happens if AI-generated content misrepresents a competitor, triggers an infringement claim through its training data, or gets published without review. That exposure flows through the cyber policy, and it requires the marketing team to be part of the AI risk conversation in a way most organizations haven’t structured yet.

The same logic applies to any function using AI in decisions that affect people: hiring, performance evaluation, access determinations, healthcare recommendations. There’s a patchwork of state-level legislation governing automated decision-making in the US, and the regulatory exposure for getting it wrong is real. These aren’t issues that sit on the CISO’s desk, but the insurance implications flow directly through the cyber and tech E&O policy. The CISO who doesn’t know about these exposures is the one who gets caught off-guard when a claim comes in.

What AI governance looks like when underwriters evaluate it

The insurance industry is working toward consistent standards for evaluating AI risk during the  underwriting process. The lack of those standards was the top concern in a recent poll of our webinar audience, ahead of policy language gaps, coverage classification questions, and attribution concerns.

What we look for right now, and what any organization should be ready to answer: Do you have an AI acceptable use policy? Do you know which AI tools your employees are actually using? Have you inventoried the AI components in your products and services? Have you thought through what model drift, hallucinations, or prompt injection would mean for your customers and your contractual obligations? Do you have humans validating AI outputs, or are systems running without human review?

That last question is one we’ve had to answer internally. At Resilience, our underwriters are required to manually synthesize their assessment of every risk. AI accelerates the analysis, but a human owns the conclusion. The model needs to be explainable: what inputs it used, how the output was reached, where a human checked it. That’s both a governance requirement and, increasingly, a regulatory expectation  and it’s the standard I’d point any organization to when thinking about how to govern AI in consequential decisions.

The market hasn’t priced this yet

Cyber insurance has been in a soft market for several years. Coverage has been broad, premiums favorable, and the competitive pressure on carriers has meant that some markets ask fewer questions to win business. That’s unlikely to change when AI-driven losses accumulate.

While not an exhaustive list,AI introduces tech E&O loss exposure from model drift or hallucinations, media liability from AI-generated content, regulatory exposure from ungoverned automated decisions and aren’t yet fully reflected in how most of the market is pricing cyber risk. When those claims start materializing at scale, coverage terms will tighten and premiums will adjust. Organizations that have already built AI governance programs, documented their controls, and had substantive conversations with their brokers about how their policies respond will be in a much stronger position than those that haven’t.

The ask I’d make of any CISO reading this: bring your broker into the AI governance conversation now, before renewal, before a loss. Walk them through how AI is being used in your organization. Ask specifically, scaled to your business, how your current policy responds to an AI-related Cyber Liability, Tech E&O, and  Media Liability claim arising from, and a regulatory action stemming from automated decision-making. If they can’t answer those questions with confidence, that’s useful information to have before you need it.

The organizations to benchmark against the ones treating AI governance as a cross-functional risk management problem, not a technical security problem. That means knowing your exposure, governing your use, and making sure the people making AI decisions include someone who understands what the financial exposure and risk transfer ability looks like.

The AI governance gap

6 Min Read