The AI Accountability Ecosystem in the Era of Language Models

Summary

This article reviews and updates the framework for accountability in AI based on accountability ecosystems. We update the framework in light of the latest developments since the release of Large Language Models for general public use. We propose three interlinked updates to the original AI accountability ecosystem: (i) reorienting the accountability ecosystem to AI infrastructure and supply chains, (ii) providing greater emphasis on outcomes monitoring and identification of issues that support decentralized system improvement, and (iii) incorporating end-user accountability given the new risks of unpredictability of language models in-the-wild. Collectively, these updates mark a shift towards accountability as distributed, continuous, and institutionalized, away from a system in which frontier AI applications can be modeled as discrete products controlled by single identifiable actors with industry-specific oversight.

The framework for an “accountability ecosystem in AI” predated the generative AI revolution with LLMs. Nevertheless, it contained relevant predictions about today’s AI landscape. Written in 2021, the paper concludes with “if we collectively fail to make progress towards these [accountability] goals, towards measurable [explainability] and implementing these accountability mechanisms, we can look forward to immature regulation operating in an unbalanced ecosystem, being either ineffective or overwhelming, and continued frustration from a civil society that lacks constructive channels through which to direct their anger and to help shape AI practice”.

Five years ago, the infrastructure layer for AI was more benign, with risks concentrated more in application development than underlying infrastructure. Those building AI applications did draw on an infrastructure layer, but often limited to cloud services for processing their own data, standard libraries to implement well-understood algorithms, or broad-based online communication infrastructures. In the early 2020s, it was reasonable for an accountability ecosystem to be built around the technical team or company developing the application, even if multiple stakeholder layers and mechanisms were required to hold the developers adequately to account.

Today’s world is very different. The era of Large Language Models created a centralized infrastructure layer driving AI applications that is far from benign. Frontier models are revised without forward-looking schedules, producing new capabilities and limitations that are only weakly understood by their owners, let alone their API-users. Design choices are more art than science, involving externally opaque decisions on training set corpora, algorithm optimization, and applicable AI principles. Three years ago, Reinforcement Learning with Human Feedback (RLHF) was touted as the main technological innovation behind ChatGPT. Since then, there has been a retreat from openness in AI research in industry, followed by confusion around the timeline for the achievement of Artificial General Intelligence (AGI).

With the above developments, the weaknesses in our collective AI accountability ecosystem remain all too apparent today. This article explores what a strong system might look like in today’s context. We argue for three interlinked adjustments to the original ecosystem argument. First, reorient the accountability ecosystem around supply chains, rather than around the development team building a given application, so as to recognize the importance of assurance markets and benchmark validation across a delivery stack. Second, apply greater emphasis on outcomes monitoring and identification of issues by stakeholders in ways that support decentralized system improvement. Third, explicitly incorporate user accountability and safeguards, given the new risks raised by jail-breaking and system unpredictability. The remaining elements of the original accountability ecosystem framework continues to be valid, with an emphasis on a tiered approach to risk management, taking into account the contributions and interaction between corporate actors, their market counterparts, civil society and government. At the same time, stronger tools are necessary to deliver on the requirements of the original proposal, as we will explore in a brief discussion on explainable AI.

Collectively, the above three adjustments mark a shift towards accountability as distributed, continuous, and institutionalized, away from a system in which frontier AI applications can be modeled as discrete products controlled by single identifiable actors with industry-specific oversight. Where the most significant risks occur in infrastructure layers, the accountability approach might draw greater inspiration from such areas as communication network infrastructure or finance. Supplier licensing, model-level certification, independent safety bodies, and incident reporting systems have a larger role to play than internal and external audit functions directed at corporate API-users.

At the same time, the ecosystem perspective from 2021 remains key to the collective approach towards AI accountability: no single mechanism is sufficient and societal layers all have a role to play, from technical teams and companies through to markets, civil societies, and governments. Bias, explainability, transparency, and informed participation also remain essential principles underpinning accountability, now magnified by issues around large-scale data security and privacy. The focus on operationalization from 2021 becomes even more important as AI technologies are already too diffusely embedded for adherence to principles to be trusted or directly monitored. Converting principles into practice continues to be important as proposed with the use of nested operational procedures: decision process documentation, internal audit, external audit, external accreditation.

This article does not question the current and potential contributions of LLMs, nor do we argue for turning back the clock. We acknowledge the value of LLMs as a very impressive engineering achievement with great potential impact to promote productivity gains across the global economy. The question is how to unlock these contributions in a safe and sustainable way.

New Accountability Issues in the LLM era

In an unregulated area of knowledge such as computer systems, or in a marketplace without accountability, a system like ChatGPT can be made available to billions of users for free without stringent quality checks, ethical certification, or reliability guarantees. In some corners, this is seen as creating a healthy disruption in the market. Google certainly felt that disruption when OpenAI released ChatGPT in late 2022. The ease with which internet-style searches were enabled by ChatGPT via system interactions in natural language put Google’s keyword-driven internet advertising business model in jeopardy. Apparently, OpenAI’s goal was to test their new system, collecting vast amounts of user data to refine their models. What followed was a BigTech race to the bottom with companies competing fiercely for user engagement, lawsuits about copyright violation, and a normalization of the idea that AI will make stuff up, in some cases with serious consequences.

LLM hallucinations have not gone away. Imperfect reliability is a fact of life outside the self-contained world of axiomatic proofs and high-assurance industries. Complex socio-technical systems are very hard to verify formally and humans make mistakes. However, incentives, oversight structures, and the use of formal methods have been shown to promote error correction processes that can be deployed to improve reliability with costs proportionate to a given use case. In cases with high impact errors, such as air travel or infrastructure security, multiple layers of automated and human oversight reduce mistakes to acceptable levels, where what counts as acceptable is continually negotiated between regulators, stakeholders, and users.

Today’s LLM architectures have not yet mastered this proportionality. ’Hallucinations’ have reduced in recent years, but it is not yet possible to invest in larger AI infrastructure and reduce errors to a pre-determined level. The only current way to achieve this is via significant human-in-the-loop checks, which largely undo the productivity gains of AI and place unrealistic pressure on human monitors. A human finding errors in a stream of AI output that is tailored to convert unreliable source text into professional-sounding language is tantamount to looking for discolored pixels in a photo. Human cognition is not optimized for this task.

Unfortunately, the stakes of LLM hallucinations are rather higher than discolored pixels and likely to increase as more individuals rely on LLMs with increasing autonomy, so-called Agentic AI, to support their decision-making. LLM’s worldwide deployment and social media share the same business model predicated on the false premise that software is free or low cost when in fact users pay with their data or with the promise to investors that high usage numbers will yield world-changing profits in the future. Back in 2021, when referring to the Testimony from a Facebook Whistleblower to the US Senate, we stated it helps “highlight serious concerns of influence on topics such as the mental health of youth exposed to social media platforms, and the use of private data to manipulate information leading up to addiction to the social media platforms.” Since then, we saw the CEO of Meta apologize in another US congress hearing to the parents of children who took their lives influenced by social media. The problem is now urgent and compounded by LLM slop, where a recent court ruling found Meta and YouTube guilty of building addictive social media platforms that harmed a 20-year old’s mental health.

Humans are in the loop, but disempowered. Take for example the idea of using various LLMs in parallel to scale up the job of therapy advice, having a human therapist in hand who is alerted to take over whenever an LLM produces an output from a pre-defined list of red-flagged outputs. As in the case of a self-driving car where the human “driver” is expected to intervene to avoid an accident, having liability in the case of an accident, despite automation complacency (as drivers gain confidence in the system, their attention to the driving task diminishes), here too the therapist isn’t empowered but is made responsible when things go wrong. Human-in-the-loop only works when the human is empowered by the interaction with the AI system and can intervene meaningfully in the decision-making process that is partially automated.

The seductive power of LLM capabilities introduces further risks of disempowerment. From students to politicians, LLM users are increasingly passing off AI-generated content as their own. European Union (EU) regulation requires developers to declare the role of AI in their tools. That does not apply to anyone using AI informally. How can we ensure, as an alternative to regulation, that people take responsibility for their decisions when they increasingly lean on AI to formulate them? Furthermore, how can we make individuals or organizations accountable in this new AI informal economy, where productive gains derive from employees augmenting their work with LLMs in ways that are undisclosed or untracked?

The highest risk (and arguably the highest opportunities) in AI are in Agentic AI, that is, LLMs with autonomy to take actions such as executing auto-generated code to collect and process data automatically to improve its own performance. At the same time, data quality has become a very valuable asset when it comes to fine-tuning or training domain-specific AI systems. In practical terms, Agentic AI poses a serious cyber-security risk to computer systems, with much of the latest research being focused on neuro-symbolic systems capable of providing assurances to various LLM outputs, e.g. by combining neural networks with symbolic theorem provers for the purpose of auto-generated code verification.

Systemic implications are far reaching now. General purpose AI models have systemic implications that are less easy to predict and manage than the narrow, domain specific applications that dominated the landscape in the late 2010s. Where models were largely developed within an industry or use case, such as healthcare, finance, or retail, the boundaries of those organizations and industries largely curtailed spillover consequences. Systemic implications today are far reaching, connecting into geopolitics and international development. The AI race, principally between the US and China, strains trade relations and introduces concerns about data sovereignty. The increased role of drones in modern warfare exacerbates pressures for autonomous AI capabilities. An ongoing dispute between the company behind the Claude LLM and the US Department of War, risking a US200M contract, relates to whether the company or the government as its client should dictate the terms of use of Claude when it comes to safeguards preventing use for mass surveillance.

The economic disruption of AI is being increasingly analyzed by governments, with particular concerns about LLMs replacing junior level staff, disrupting the lifecycle of expertise transfer and skills ladders. As pointed out, when OpenAI released ChatGPT, RLHF was promoted widely by the company’s leadership as being its major technical innovation. We now know that RLHF involved exploiting, for the purpose of model alignment, a very low paid global workforce to perform data labeling watching the worst possible content on the internet for many hours a day.

Increasing levels of autonomy in AI systems with access to bank accounts, emails and calendars introduce the possibility of machines operating as independent actors in society and a range of risks that would seem outlandish if it were not for the rapid progress of recent years. A small but fast growing set of AI users are calling for independent rights and protections for AI systems, often grounded in a belief that AI systems are conscious or should be considered moral actors given their apparent agency. What will happen when these demands grow in volume and become the focus of political campaigns?

Implications for the AI Accountability Ecosystem

The AI accountability ecosystem argued that the then-accountability ecosystem was heavily imbalanced, being a factor in its inability to translate intensifying civil society concerns into productive changes and engender more positive attitudes towards AI. The then-ecosystem was skewed towards high-level principles (over 100 at the time) and disempowered tag-on external accountability mechanisms (such as senior boards of external observers). These external mechanisms proved unable to drive change internally, so those involved raised high-profile public concerns instead, further alienating internal processes. An explicit ecosystem approach was suggested as a way to correct these issues, with attention to developing multiple layers of interaction from actors and companies through to market counterparts, civil society, and governments.

Many of the same issues apply even more intensively now than five years ago. The traditional strong arm of modern externally-led accountability seems to matter little in the face of LLM’s technical, financial, and geopolitical reputation. In principle, society’s strongest accountability systems are independent members on corporate boards, government hearings, and political oversight. This does not seem to work in the case of disruptive AI, as best described by a former OpenAI board member.

Next, we argue that AI ethics cannot be an afterthought. It needs to be a requirement considered from the start, evolving with model development, properly documented, stress-tested and ideally formally verified, a requirement to be achieved by design. Nevertheless, the AI race is on. While an AI regulatory agenda promoted some coordination around AI safety, it didn’t seem to have produced any concrete outcomes, and the risks to computer systems’ security has increased. We will discuss how an updated AI accountability ecosystem should address eac…

Based on the paper, here are the specific improvements I can implement in an AI system, along with what the improved system can do:

Improvement 1: Supply Chain Accountability Layer

• What I implement: Add a structured, machine-readable accountability ledger that tracks model provenance, training data lineage, fine-tuning history, and contractual obligations across the entire delivery stack (frontier model → API intermediary → integrator → deployer).

• What the improved AI system can do: Automatically generate and verify compliance certificates at each layer, flag when a downstream application uses an upstream model beyond its documented intended use, and enforce contractual constraints (e.g., prohibiting population-level monitoring) via runtime policy checks.

Improvement 2: Continuous Outcomes Monitoring with Tiered Risk Escalation

• What I implement: Integrate a real-time monitoring module that logs user interactions, system outputs, and downstream consequences. This module classifies events into risk tiers (low, medium, high, critical) and triggers structured escalation pathways (e.g., automated alerts to internal auditors, external assessors, or regulatory bodies depending on severity).

• What the improved AI system can do: Detect emergent harmful behaviors (e.g., hallucination-induced misinformation, jailbreak attempts, unintended inference of sensitive attributes) within minutes of occurrence, log them with full context, and automatically initiate mitigation workflows—such as rolling back a model version, restricting access to specific prompts, or notifying the responsible supply-chain actor.

Improvement 3: User Accountability and Co-Production Safeguards

• What I implement: Add a user-behavior monitoring and intervention layer that tracks adversarial prompting, jailbreak attempts, and misuse patterns. This layer includes friction mechanisms (e.g., warnings, time-outs, mandatory educational prompts) and records user accountability signals (e.g., terms-of-service acceptance, usage agreements) in a tamper-evident log.

• What the improved AI system can do: Distinguish between accidental misuse and deliberate adversarial behavior, apply graduated sanctions (e.g., temporary feature restrictions, permanent bans) without shifting blame from developers, and generate audit-ready reports on user-driven system vulnerabilities for external review.

Improvement 4: Neurosymbolic Verification for High-Risk Outputs

• What I implement: Incorporate a neurosymbolic verification module that combines the LLM with a symbolic theorem prover or knowledge-graph reasoner. This module checks high-risk outputs (e.g., code generation, medical advice, legal analysis) against formal rules and domain constraints before final delivery.

• What the improved AI system can do: Reduce hallucination rates in high-stakes domains by rejecting or flagging outputs that violate logical consistency or known facts, provide formal proofs of correctness for generated code, and enable human-in-the-loop intervention with clear, explainable error reports—rather than relying on human monitors to spot subtle errors in fluent text.

Improvement 5: Explainability and Audit-as-Code Integration

• What I implement: Add a continuous assurance framework that maps governance requirements (e.g., EU AI Act, NIST AI RMF) to auditable, machine-readable rules. This framework automatically generates audit trails, documentation, and compliance reports at every stage of the model lifecycle.

• What the improved AI system can do: Produce real-time, standardized audit logs that external auditors can verify without manual documentation review, automatically flag non-compliance with regulatory or contractual obligations, and provide stakeholders (including civil society) with transparent, queryable access to model behavior and risk assessments—reducing the opacity that currently hampers external accountability.

Summary of Capabilities of the Improved AI System:

• Proactive risk detection across the supply chain, not just at the application layer.

• Real-time, evidence-based accountability with structured escalation to the right actor.

• User-aware safeguards that balance developer responsibility with user co-production.

• Formal verification for high-risk outputs, reducing reliance on fallible human oversight.

• Transparent, auditable operations that support both regulatory and non-regulatory governance mechanisms.

These improvements collectively shift the AI system from a discrete, opaque product to a distributed, continuously monitored, and institutionally accountable socio-technical ecosystem—exactly as the paper advocates.

This article reviews and updates the framework for accountability in AI based on account- ability ecosystems. We update the framework in light of the latest developments since the release of Large Language Models for general public use. We propose three interlinked updates to the original AI accountability ecosystem: (i) reorienting the accountability ecosystem to AI infrastructure and supply chains, (ii) providing greater emphasis on outcomes monitoring and identification of issues that support decentralized system improvement, and (iii) incorporating end-user accountability given the new risks of unpredictability of language models in-the-wild. Collectively, these updates mark a shift towards accountability as distributed, continuous, and institutionalized, away from a system in which frontier AI applications can be modeled as discrete products controlled by single identifiable actors with industry-specific oversight.