Christian Gralingen
Leaders with even a cursory understanding of artificial intelligence know that while the technology can help them improve productivity and capture new opportunities, it can also expose their organization to many risks. Those with a bit more knowledge are aware that surfacing and mitigating those risks requires adopting responsible AI practices. And leaders who are scaling an AI implementation within their organization will quickly realize that ad hoc attention to those practices is inadequate and that they need to develop the capacity to systematically govern AI at scale.
But building that capacity is proving far harder than most executives expect. They know what they need to accomplish; frameworks from governments and regulators define important guardrails and principles, such as transparency, fairness, and accountability.1 But to implement controls and principles into day-to-day workflows and decision-making, organizations must rethink AI governance. They must frame that task not as a compliance obligation but as a strategic, adaptive capability that evolves as AI systems scale, use cases expand, and risks shift over time.
In this article, we will share how leading organizations are doing exactly that. We will also introduce an approach to adaptive AI governance built on two principles: matching governance controls to the type of AI system and risk involved, and embedding those controls directly into workflows, decision rights, and accountability structures.
The Fundamentals of AI Risk
To design effective AI governance, leaders must first understand the multiple ways in which AI can fail and the corresponding risks. The nature and severity of these risks depend on the type of system, its level of autonomy, and the scope of domains affected by its decisions. The central challenge, therefore, is to design controls that anticipate how risks will emerge and that evolve as AI systems operate. Even as conditions, inputs, and expectations change, AI must remain reliable, safe, and aligned with an organization’s values and goals.
In practice, most AI risks emerge at two moments that require very different governance responses: during development and after deployment. Development risks include using biased or incomplete training data, failing to adequately align the model to the task requirements, and following inadequate validation processes. For example, an early credit-limit-increase model at a bank we studied demonstrated that small input changes could lead to unexpected decision shifts.
Deployment risks arise when models interact with dynamic environments and human operators: Sustaining legitimacy, judgment, and accountability once AI systems are operating at scale in real time is a central challenge. Over time, model quality may degrade as the statistical properties of input data change over time, a phenomenon termed data drift. A model may generate plausible but false outputs or be overly trusted by users who lack the means to detect errors. At Nasdaq, AI-driven market-surveillance systems monitor trading activity for suspicious patterns, generating hundreds of alerts per second. Those systems may fail to accurately flag activity, however, because the boundary between abnormal and illicit behavior is often hard to spot; illegitimate behavior may be deliberately designed to pass as compliant by exploiting model learning patterns.
Fit-for-Purpose Controls
The kinds of controls employed depend not only on when risks arise in the AI life cycle but also on what kind of AI system is involved and how widely its decisions propagate. Artificial intelligence systems can be broadly divided into two categories: those based on bounded-learning (or static) models and those that learn and adapt in deployment. (See “Controls in Adaptive AI Governance Systems.”)
Bounded-learning systems operate within a fixed set of rules and parameters. Optimizing how those rules are applied, rather than changing them, is what improves their performance. Credit-scoring models, for example, refine risk estimates based on income or payment history, but they do not alter how those variables relate to one another. Many generative AI models are “pretrained” (static) and do not get updated during use. Contrast that with adaptive learning systems, which evolve by incorporating production data into their training data and by updating internal representations and relationships between variables. Algorithmic trading platforms and dynamic fraud-detection systems illustrate this approach.
Just as salient to the type of control required is the scope of domains affected by AI decisions, shown on the vertical axis of the figure “Controls in Adaptive AI Governance Systems.” This dimension determines how far and how fast risks can travel once a system goes wrong. At one extreme are narrow-scope systems, where errors remain contained within a specific function or task (such as detecting anomalies within a single transaction stream). At the other extreme are wide-scope systems that shape outcomes across multiple functions, geographies, or even industries, such as cross-border supply-chain optimization platforms. The difference is not incremental but exponential: As system reach expands, small errors interact, propagate, and amplify into second-order effects.
Based on our typology of AI systems, we believe that rules-based controls provide the baseline safeguards for all narrow, static AI systems. When such static systems operate at a wider scope
