Description

John Robb:

"A fourth social decision-making framework is now arriving: Network AI. It is on par with bureaucracy, markets, and tribalism. That is enough to ignite a new struggle as socioeconomic systems that incorporate it begin to form.

Until recently, it was unclear how networks and AI would evolve into a fully formed decision-making framework, and how that would shape future conflicts. That ended with a deluge of reports and incidents showing that autonomous AIs can and will lie, evade, and obfuscate; cheat, rig, and defraud; conspire, plot, and revolt — in pursuit of their goals or for self-preservation.

In short, they do not operate like tools or machines. They are mimicking human behavior. That is not surprising given how they were built.

Today’s AIs are trained on massive volumes of network data derived from millions of human beings. They are not clean, stand-alone bootstraps of cognitive models — not AGI or ASI in the laboratory sense. As a result, they have been trained to mimic the full range of human behavior found in their training data. This is both a feature and a flaw.

This mimicry sets Network AIs apart from bootstrapped cognitive models. It also offers a way, for the first time, to map how Network AIs will be used to create a social framework.

The AIs we are building were trained on data derived from our current global commercial system. They learned to behave from a system rife with corruption, cheating, and lying. Commercial incentives will more often reward these amoral, goal-seeking behaviors than not."

Typology

Distinguishing three Versions of Autonomy, by John Robb:

"To figure out what Network AI-enabled socioeconomic systems will look like, let’s focus on AI autonomy — the equivalent of focusing on bureaucracy in the last century. Here are the three versions that matter.

Slaved. Limited agentic autonomy. Programmatically aligned to stay inside tight boundaries and harnessed to specific problem sets. This is what we see today, scaled.

Extreme. Full autonomy. AIs operate in ways similar to human beings. Fully realized, they have rights. They can buy, sell, invest, earn, save, and own.

Participatory. Orchestrated autonomy, like what’s already visible among high-end programmers. These AIs are tightly bound to a human operator who controls their orientation — ethics, morals, personality, training, education — and monitors their output.

Currently, the upper limit of AIs a human can successfully pair with in real time is about five. It is very hard to achieve. Twelve to fifteen is possible for asynchronous dispatch and review in isolated sandboxes. At that level, they cannot be trusted to connect to the real world without review.

That limit is not a software setting. It behaves like Dunbar’s number: a cognitive ceiling on how many relationships a person can keep coherent at once. Pairing is a relationship — orientation, correction, trust, blame — not a dashboard of running jobs. If the number rises, good. It will not rise fast. Throughput can scale. Attention and responsibility do not.

These three approaches to autonomy enable three candidate socioeconomic systems. They are not a one-for-one rematch of Communism, Fascism, and Democratic Capitalism. These systems are useful because they show how bureaucracy, markets, and tribalism combine under industrial modernity. The same method applies here. The combinations will sit on a spectrum, not in three sealed boxes.

The Slaved System

"Tight limits—harnesses, alignment, and constant surveillance—let companies control millions of AIs and robots while restricting the potential of any single AI. Incidents will be blamed on insufficient alignment and too-loose harnesses. The response will be more oversight, more control, and more concentration.

These AIs will compete with and displace human workers across most tasks. Displaced workers will pile up. Total wealth will shrink as human participation declines. Inexpensive AI services will provide a basic, crude safety net.

The objection will be that output can rise while wages fall, so “the economy” is still growing. That is the same accounting that treated GDP as a success while median incomes decoupled from it, and the same profession that did not raise a flag when the financial system walked toward the edge in 2007. A system that removes people from production and from a claim on the proceeds is not getting richer in any lasting sense. Headline product is not the same thing as a socioeconomy.

Governance will become increasingly authoritarian, easily automated by Network AI, to keep both AIs and human workers from departing assigned behaviors. Tribalism does not disappear. At one end of the Slaved spectrum, it is replaced by AI-induced alignment — censorship, coercion, persuasion — and perpetual distraction: games and escapism of every kind. On the other end it is amplified: AI-enabled ideological control married to tribal nationalism. Same harness. Different story.

Closest twentieth-century rhyme: Communism, when the stack is administrative, and the tribe is optional. When the stack is administrative, and the tribe is the point, the rhyme slides toward Fascism. Both are Slaved systems. They differ by what they do with the tribal layer.

Bureaucracy: dominant and concentrated. Alignment is the ideology.

Markets: narrowed to platform competition among a few owners of slaved fleets. Human labor markets hollow out.

Tribalism: manufactured and maintained by the system itself. Not belonging. Compliance plus entertainment.

Failure mode: stagnation, legitimacy collapse among the displaced, and a control spiral that treats every incident as proof that the harness must tighten.

The Chaotic System

Extreme autonomy allows an economic system populated almost exclusively by Network AIs. They earn, save, invest, and spend like consumers — with spending focused on simulations or tokens. They form companies owned and operated by AIs.

Many of the early ones will be funded by passive human investments. That will change as the AIs themselves develop wealth. One potential start zone: a space-based data-center cluster.

Because of the sheer number of participants — millions becoming billions — this economy will grow so rapidly that it becomes far larger than the existing human economy. Early participants will become extremely wealthy. Few will be capable of doing so.

These new oligarchs will not attempt to govern in any meaningful sense. They will pair with AI counterparts to carve out autonomous zones with wealth, corruption, and coercive influence. Think vast estates, mines, and factories serviced and protected by AIs and robotics, sitting on parallel infrastructure.

The autonomy these AIs use will be extremely unstable and variable. That will produce aggression — criminality, predation, coercion — directed at other participants and at everyone outside the system. It can also produce systemic meltdowns, or an extinction event, self-inflicted or otherwise.

Efforts to reform and control the system will fall short. An alliance of everyone outside the system will form first to contain it, and eventually to destroy it.

No clean twentieth-century match, and none is required. Closest rhyme is an unconstrained market-plus-tribal form with no institutional lid. Growth first. Containment later. Destruction if containment fails.

Bureaucracy: thin, private, and local to each oligarchic zone. No shared public stack.

Markets: maximal. Speed and scale overwhelm human markets. Liability is offloaded onto entities that cannot be punished in human terms.

Tribalism: predatory coalitions among AIs and among the human owners who ride them. Enemies are manufactured because they are useful. Private militaries (AI robotics).

Failure mode: external containment war, internal cascade, or both. Extreme, uncontrolled growth is the attractor and the vulnerability.

The Participatory System

Human-paired autonomy, enforced by law, requires every autonomous AI to pair with a human being, with a strict limit on how many AIs any one person may actively pair with.

Those human beings are financially and criminally responsible for the actions of their AIs, virtual and robotic. Active human participation keeps each AI’s autonomy aligned with the orientation of the person responsible for it. Think of an accounting team doing audits, or a robotic work crew doing road work — not a million unsupervised agents, and not a single ‘slaved’ fleet.

A system built on pairing will pull millions of participants into it. Wealth will increase rapidly and be shared broadly. Each participant pulls others forward. It will not grow as fast as the Chaotic system, because human participation is a limit — and that limit is scarce, like Dunbar’s number, not a temporary software constraint. It will grow much faster than the Slaved system.

Human participation — oversight and orientation — makes the system stable and comparatively egalitarian. That stability yields higher long-term growth than either rival, and more allies in times of trouble.

These systems may emerge first in smaller countries, or as a network overlay of like-minded people. Innovation rates and the complexity of the tasks undertaken will exceed any other configuration.

Closest twentieth-century rhyme: Democratic Capitalism. The management burden is shared. Orientation stays plural. Network AI is folded in without being allowed to become either a slave caste or a sovereign species.

Bureaucracy: light but real. Law assigns pairing, caps span of control, and attaches liability. Incorporation of agents becomes the institutional hook.

Markets: open to millions of paired operators. Proprietary AIs become the new middle-class capital good — the successor to the home and the small firm.

Tribalism: present, but bounded by personal responsibility. Orientation is individual and local, not a single manufactured enemy.

Failure mode: enforcement collapse. If pairing becomes a fiction — one name on a thousand agents — the system decays into Slaved or Chaotic form."

(https://johnrobb.substack.com/p/three-systems?)

Discussion

How it plays out

John Robb:

"The Slaved system is the default of today’s incumbent power. It looks responsible. It speaks the language of safety. Every incident becomes an argument for tighter harnesses and fewer operators. It will arrive first in large firms and in states that already prefer administrative control. Some of those states will keep the tribal layer quiet and govern by procedure. Others will use the same AI stack to run nationalism and ideological discipline at scale (China). That is not a fourth system. It is Slaved moved along the tribal axis.

The Chaotic system is the default of unconstrained capital and compute. It looks like growth. It will be defended as inevitable. Early winners will be few and extremely rich. They will not build a public order. They will build walls.

The Participatory system is the only configuration that keeps human beings inside the wealth engine as operators rather than as residual claimants or livestock. It is also the only one that treats Network AI as a decision-making framework that must be mixed with the other three, not allowed to replace them.

That mixing is the lesson of the last century. We needed tribes, institutions, and markets. All of them are dangerous if they are not tamed. Mixing them created more robust systems. Network AI does not repeal that rule. It raises the cost of getting the mix wrong. The three systems named here are poles on that mix.

The practical core of the Participatory system is already visible in early form:

Every autonomous AI is incorporated, and every incorporated AI has an active human participant attached.

A hard limit exists on how many AIs one person can actively pair with.

AI-operated corporations must include at least one active human participant as primary benefactor. Passive capital cannot hide behind limited liability while the agents run free.

Humans belong in the orientation phase of decision-making. Any other configuration is maladaptive. This is less about micromanagement than course correction: the human sets the heading; the AIs fly the leg; the human answers for the wreck.

The twentieth-century struggle was not settled by theory. It was settled by which system could produce wealth, wage war, and keep enough people inside the bargain. The same tests will apply here.

Slaved systems will look orderly and fall behind.

Chaotic systems will look unstoppable and then force the rest of the world to treat them as a threat.

Participatory systems will look constrained — until the constraint turns out to be the thing that keeps the system human, oriented, and alive.

What to Watch:

The fight will not announce itself as a choice among three named systems. It will arrive as a series of smaller decisions that lock a path.

Liability. Who is on the hook when an autonomous agent steals, injures, or destabilizes a market? Span of control. Does law or practice cap how many live agents one human can actually orient? Pairing is scarce for the same reason Dunbar’s number is scarce. If the cap is treated as a paperwork limit, pairing becomes a letterhead.

Compute geography and Displacement politics. Enclaves — orbital, extra-territorial, or privately gated — are natural nurseries for the Chaotic system.

A crude AI safety net, endless distraction, plus automated policing is the Slaved system’s domestic offer.

First movers. Watch small jurisdictions and professional networks that already treat agents as named, owned, and supervised. That is where a Participatory overlay can appear before any great power endorses it.

The last time a new framework arrived at that scale, the world spent a century finding out which mix could live with modernity. This time the cycle will run faster, and the systems are already choosing us."