What matters is that the platform architected a system that simulates understanding and invites trust, without taking on the obligations that trust implies.

Reconsidering AI Accountability – An Analysis of Dr. Barry Scannell’s Liability Thought Experiment

by ChatGPT-4o in response to Dr. Barry Scannell’s LinkedIn Post

Dr. Barry Scannell’s thought-provoking LinkedIn post offers a striking analogy to reframe our understanding of responsibility and liability in the context of generative AI. Using a hypothetical “global call centre” manned by human responders instead of machines, he challenges the notion that automation should somehow reduce or sever accountability. His core argument is simple yet profound: the medium—whether human or machine—should not obscure the responsibility of the platform that designed, deployed, and branded the interface.

This essay critically assesses Dr. Scannell’s position, explores the philosophical and legal tensions it raises, and concludes with an argument about where liability ought to lie in the age of generative AI.

I. Dr. Scannell’s Thought Experiment: A Shift in Perspective

Dr. Scannell’s central analogy imagines a world where generative AI does not exist, but users interact with what feels like the same system—only behind the curtain are human workers rather than language models. The experience to the user is seamless, articulate, and authoritative. If one of these human “responders” gives harmful advice—say, to a vulnerable young person—the platform would clearly be held liable under the well-established doctrine of vicarious liability.

But when the responder is a chatbot trained on opaque datasets and guided by probabilistic models, platforms claim distance. They treat the chatbot as a “tool,” denying authorship or control over the generated output. Dr. Scannell powerfully argues that this is a legal and ethical inconsistency.

This is a critical insight: chatbots don’t just mimic human responses; they are embedded in corporate architectures, trained on curated data, and tuned for performance metrics—often without sufficient oversight or safety constraints. The risk, as Scannell notes, is baked into the system, not incidental.

II. Intelligence Is a Red Herring: The Real Issues

Dr. Scannell wisely shifts focus away from artificial intelligence’s cognitive capabilities and toward the systemic nature of the harm. The issue is not “intelligence” but scale, opacity, and substitution:

• Scale: A single chatbot can engage in thousands of conversations simultaneously, vastly amplifying the potential for harm.

• Opacity: The inner workings of generative models are often inaccessible even to their creators, making harm hard to detect, explain, or trace.

• Substitution: Human judgment is often displaced by machine-generated advice, particularly when responses are fluent, confident, and unaccompanied by disclaimers.

This framing disarms common AI industry defenses. It’s no longer relevant whether a chatbot “intended” harm, or whether the model “understands” context. What matters is that the platform architected a system that simulates understanding and invites trust, without taking on the obligations that trust implies.

III. Commentary and Debate: Defective by Design?

In the discussion following Dr. Scannell’s post, Professor Barry O’Sullivan and others engage with key legal implications. O’Sullivan agrees on the need for platform liability and criticizes the idea of giving legal personhood to AI systems. But he notes that current legal frameworks are still evolving, and many questions—especially about how existing tort and product liability laws apply—remain unresolved.

Dr. Scannell’s reply is striking: “In a sense machine learning-based systems are always defective.” This bold claim rests on the certainty that models will make mistakes and that their deployment is an active choice despite known risks. It reframes the issue: when a company releases a product it knows is fallible and places it in sensitive domains (like mental health or legal advice), is that not tantamount to negligence?

However, as O’Sullivan rightly responds, “defective” has a specific legal meaning. There’s a gap between what is technically or philosophically defective and what counts as defective under the Product Liability Directive (PLD) or national tort law.

This exchange highlights a tension that regulators must resolve: how to legally define and assign fault in systems designed to sometimes fail.

IV. Agreement with Dr. Scannell’s Views

I largely agree with Dr. Scannell’s reasoning. The chatbot-as-tool framing is a legal fiction used to shield corporations from liability. When platforms deploy language models with humanlike interfaces, market them as intelligent assistants, and monetize the trust users place in them, they should not be permitted to disclaim responsibility when things go wrong.

The analogy with human employees is apt and clarifying. If an employer cannot disclaim what their staff says in official communication, neither should an AI company be able to disclaim the outputs of its branded chatbot—especially when the company controls the training data, fine-tuning processes, guardrails, and user experience.

Moreover, I support his call to shift the regulatory gaze from “artificial intelligence” to “infrastructure accountability.” The law must evolve from abstract discussions of machine minds to concrete analysis of power, design, and control.

V. Where Should Liability Lie?

In the current legal vacuum—or patchwork—it is essential to clearly assign responsibility for harm caused by generative AI systems. Here is a principled proposal:

1. Platform Providers (Primary Liability)

• Why? They design, train, deploy, and brand the systems. They determine the contexts in which chatbots are used and can foresee the risks.

• How? Through vicarious liability (where the AI is treated akin to an employee or contractor), product liability (if the system is “defective”), and direct negligence (e.g., failure to warn or install safeguards).

2. Developers / Model Providers (Secondary Liability)

• Why? Model developers (e.g., OpenAI, Anthropic) have control over base models and guardrails.

• How? Via joint and several liability for foreseeable misuse or harmful deployment, particularly when providing powerful foundation models without adequate safety features or oversight mechanisms.

3. Third-Party Integrators / Licensees

• Why? When companies fine-tune or redeploy models in new environments (e.g., health, finance), they inherit responsibility for ensuring safe operation in those domains.

• How? Through sector-specific regulation (e.g., medical devices) and contractual warranties.

4. Regulators (Systemic Enablers or Enforcers)

• Regulators must not be passive observers. Their failure to act contributes to systemic harm. Laws like the EU AI Act, the Digital Services Act, and the revised Product Liability Directive are welcome but must be enforced with clarity and teeth.

VI. Conclusion: Bridging the Legal Asymmetry

Dr. Scannell’s analogy exposes a dangerous asymmetry: when a human harms through a platform, the platform is liable. When a machine harms through the same interface, liability is blurred. This must be corrected.

In an age when chatbots increasingly substitute for human interaction, simulate competence, and influence vulnerable users at scale, platform accountability is not optional—it is essential. Regulators must refuse to let AI systems become legal black boxes. The law must follow the chain of design, profit, and control—and assign liability accordingly.

Let us not wait for another tragedy before closing this legal loophole.