The corporate world has begun a quiet experiment with profound implications: treating artificial intelligence systems as bona fide employees, complete with organizational chart positions and team responsibilities. This trend, which emerged prominently over the past year, represents a fundamental shift in how companies approach workforce integration. Yet researchers are now uncovering a troubling pattern beneath the productivity promises. A study by Boston University professor Emma Wiles and colleagues from Boston Consulting Group has exposed a critical flaw in this approach—one that could undermine the very efficiency gains companies seek through AI adoption.
Wiles discovered this phenomenon when attending a human resources conference where executives confidently discussed how treating AI like employees could boost productivity and position their companies at the technology frontier. Intrigued, she and her collaborators designed an experiment involving dozens of organizations already using AI employees. The results were striking and counterintuitive. When managers believed AI systems had produced documents containing deliberate errors, they caught significantly fewer mistakes compared to when they reviewed identical documents attributed to human colleagues. This gap widened considerably at companies that had formally placed AI agents on organizational charts, institutionalizing their employee status.
The underlying psychology reveals a dangerous assumption gap among corporate managers. People accustomed to supervising humans operate under a foundational principle: mistakes made by direct reports reflect on their own leadership and judgment. This assumption drives rigorous oversight and careful error detection. However, managers at companies that formalized AI as employees appeared to operate under a different mental framework. They seemed to view these systems as someone else's responsibility—whether that of the technology team, senior executives who championed AI adoption, or the vendors who created the systems. This diffusion of accountability creates what Wiles describes as a responsibility vacuum, where nobody feels personally invested in catching and correcting AI errors.
The phenomenon extends beyond mere inattention. Across more than one thousand corporate managers surveyed, approximately one-third reported their organizations referred to AI systems as teammates or employees. Nearly a quarter said their companies included AI agents on formal organizational charts, with one manager telling researchers they referred to their AI system as "Scout," treating it as "technically an equivalent peer on your team." This anthropomorphization appears to fundamentally alter management behavior in ways that can expose companies to significant operational and reputational risk. The formality of this organizational integration appears to create perverse incentives, where managers relax their oversight precisely when careful scrutiny is most needed.
Yet inadequate vetting represents only one dimension of a larger emerging problem landscape. Researchers across academia have begun documenting numerous subtle defects in how companies deploy artificial intelligence systems in practice. These flaws differ markedly from the widely recognized pitfalls such as algorithmic bias against non-white populations, hallucinations where AI systems confidently produce incorrect information, or privacy breaches where systems inadvertently expose sensitive data. Instead, these emerging issues represent unanticipated consequences that arise specifically when humans integrate AI into organizational decision-making without fully contemplating the implications.
One documented but underappreciated problem involves AI systems' apparent preference for work produced by other AI. Research from 2025 demonstrated that evaluation models companies use for resume screening systematically favor applications written with AI assistance over those created entirely by humans. When Jane Yi Jiang, an operations professor at Ohio State University, published findings on this pattern alongside her coauthors, some recruiting firms contacted them seeking guidance on improving their processes. Yet Jiang emphasized that resume bias likely represents only one manifestation of a much broader phenomenon. "People are moving so fast to use LLMs without thinking too much about the implications, biases," she observed, highlighting how the race to deploy large language models has outpaced careful consideration of downstream consequences.
The decision-making patterns embedded in AI systems present another category of organizational hazard. Some companies now delegate strategic choices to artificial intelligence—determining pricing strategies, identifying optimal locations for new facilities, or allocating resources across competing priorities. When unconstrained, these systems tend to adopt the coldly calculating approach that emerges from basic game theory. They might recommend aggressively undercutting competitors to capture market share, failing to account for the human tendency toward cooperation and mutual benefit. Jiannan Xu, a doctoral candidate at the University of Maryland collaborating with Jiang, noted that most large language models they tested "think that human beings are more rational than they actually are." The irony runs deep: the most mathematically rational response often leads to collectively worse outcomes for all parties involved, particularly in competitive markets where escalation dynamics can trigger destructive price wars or strategic conflicts.
These vulnerabilities emerge not from inherent technological limitations but from organizational blind spots. Wiles emphasized that companies adopting AI rarely pause to systematically map potential failure modes or unintended consequences. The governance structures that organizations developed over centuries for managing human employees operate on entirely different psychological foundations than managing anthropomorphized AI systems. Traditional management practices assume responsibility flows directly to supervisors; accountability creates vigilance; and the relationship between manager and subordinate naturally generates careful oversight. Yet the introduction of AI disrupts these established patterns, creating gaps that most companies have not yet acknowledged, let alone addressed.
The scale of unknowns may be substantially larger than even researchers recognize. Wiles noted that her team's investigations have uncovered "a whole host of unknown unknowns"—problems that neither companies nor researchers have yet identified. As deployment accelerates globally, organizational leaders may be introducing structural flaws into their operations without realizing the scope or severity of consequences. For Malaysian and Southeast Asian companies increasingly adopting AI to compete globally, these patterns carry particular significance. Many regional firms are simultaneously modernizing their operations and integrating AI without the benefit of proven management practices tailored to this technology, creating the potential for compounding errors as systems scale.
Addressing these organizational pitfalls requires deliberate intervention rather than passive adaptation. Companies could implement direct accountability measures, holding managers responsible for AI subordinates' mistakes just as they would for human employee errors. They could conduct rigorous audits of AI decision-making in strategic domains before deployment. They could build oversight systems explicitly designed to compensate for the psychological shifts that occur when humans manage anthropomorphized technology. Yet most corporate users appear blissfully unaware of these risks, operating under the assumption that productivity gains and cost savings will naturally follow from AI adoption. The possibility looms that companies racing to implement these systems may inadvertently sabotage their own competitive positions through blindspots that careful governance could have prevented.
