Yiley Shao on Machine Agency vs Autonomy

From P2P Foundation Wiki
Jump to navigation Jump to search

Discussion

Synthesis by ChatGPT, prompted by Michel Bauwens, and inspired by Chor Pharn's reference [1] :

Shao Yilei (邵怡蕾; Yilei Shao) is a Chinese scholar and professor at East China Normal University, where she is associated with the Shanghai AI-Finance School. She holds a PhD in computer science from Princeton University and works across artificial intelligence, economics, governance, education and the social consequences of technological change. Her recent writing examines how increasingly capable AI systems alter the relationship between human decision-making, institutions and governance. In 2026 she published work arguing that AI is creating a widening gap between the speed of technological change and the slower adaptation of governance and social institutions.

A useful distinction in Shao's discussion of AI systems is between machine agency and machine autonomy. The terms are related but describe different properties. Autonomy concerns the extent to which a system can operate without continuous human intervention. An autonomous AI agent can receive a broad objective, determine intermediate steps, use tools, respond to changes in its environment and continue executing a task without requiring a human to approve every action. Autonomy is therefore primarily a question of operational independence.

Agency concerns the capacity of a system to function as an actor within a wider social or institutional environment. In the context of AI, this can mean the ability to initiate actions, influence outcomes, interact with other actors and participate in processes that previously depended directly on human decision-makers. Agency therefore concerns not simply whether a machine can execute a task independently, but whether its actions have become consequential within a human system. Contemporary research similarly distinguishes operational autonomy from stronger questions about agency, authority and responsibility.

The distinction can be illustrated by an AI system instructed to negotiate with a supplier. If the system independently chooses the sequence of emails, searches for information, compares offers and conducts the negotiation, it possesses a significant degree of autonomy: the human does not direct each individual step. But the system's agency depends on the institutional role it has been given. If its actions can commit the organization to contracts, allocate resources or alter relationships with outside parties, the machine has acquired a meaningful role as an actor within that organization. It may therefore exercise agency in a functional sense without possessing human-like consciousness, intentions or moral responsibility.

This distinction is particularly important because autonomy can increase without transferring ultimate authority. An AI may be highly autonomous while still pursuing objectives, constraints and standards established externally by humans or institutions. The system can determine how to achieve an objective without determining which objectives ought to be pursued. This is one reason contemporary discussions of agentic AI increasingly distinguish planning, tool use and autonomous execution from questions of authorship, authority and responsibility.

Shao's broader work places this problem within a theory of AI-era governance. She argues that AI systems are becoming capable of executing increasingly complex tasks in domains such as education, finance, research and administration, while existing institutions and accountability mechanisms adapt more slowly. She therefore emphasizes the importance of maintaining human capacity to audit algorithms, question AI-mediated decisions and participate in the governance of technological systems. In this view, the important question is not simply whether AI can act autonomously, but how human agency and institutional responsibility are preserved when increasingly autonomous systems become part of social decision-making.

The distinction is relevant to Chor Pharn's A Republic Capable of Betrayal. Pharn uses the emerging literature on machine agency and autonomy to push the AI-native-organization argument further: if AI systems can act with substantial operational independence, organizations may eventually contain large numbers of machine actors performing research, administration, commerce and coordination. This could change the relationship between organizational size, human labour and political institutions. The further political conclusions about the weakening of locally embedded economic actors and the future of the republic are Pharn's extrapolation rather than claims that should be attributed directly to Shao.

The broader significance of Shao's distinction is that AI automation is not simply a question of replacing human tasks. As machines become capable of initiating and coordinating actions, the relevant question becomes how much agency is being delegated to them, how much autonomy they possess in exercising that agency, and where ultimate authority and responsibility remain. This provides a conceptual bridge between the technical development of AI agents and the institutional questions raised by AI-native organizations.

Sources

Retrieved from "https://wiki.p2pfoundation.net/index.php?title=Yiley_Shao_on_Machine_Agency_vs_Autonomy&oldid=150546"