Responsible AI Means Knowing the Limits of Agent Autonomy
72% of a 29-expert AI-governance panel agree that treating AI agents as autonomous decision-makers is a governance failure mode, because it lets the humans and institutions behind them evade legal and moral accountability.
Summary
The fifth installment of MIT Sloan Management Review and BCG's joint Responsible AI expert panel series surveys 29 AI governance experts on whether treating agents as autonomous decision-makers constitutes a governance failure; 72% agree or strongly agree. Panelists distinguish operational autonomy (agents that plan, call tools, and act without constant oversight) from moral or legal accountability, which they argue agents cannot hold, citing the Moffatt v. Air Canada tribunal ruling rejecting a company's attempt to treat its chatbot as a separate accountable entity. The authors offer five recommendations: calibrate autonomy to the stakes rather than to capability, enforce limits by system design rather than policy alone, name an accountable human for every decision before deployment, govern the surrounding sociotechnical system rather than the agent itself, and build a culture where employees are rewarded for challenging agent outputs rather than penalized for disclosing AI use.
Editor's opinion
Well-evidenced AI-governance analysis: a quantified 29-expert panel (72% agreement, full distribution reported) rather than anonymous testimony, genuine dissent included rather than smoothed over, and a real legal citation (Moffatt v. Air Canada) grounding the accountability argument. The "operational autonomy vs. moral/legal accountability" distinction is sharp and well-argued, and the five recommendations are concrete design choices rather than vague principles. A reader comment on the page raises a real unresolved tension worth sitting with: naming an accountable human for every agent decision doesn't remove the review burden, it relocates it, and reviewer capacity won't scale with agent-decision volume the way agent deployment will.
Key concepts
Operational vs. moral/legal autonomy · sociotechnical governance · accountability-by-design