What is AGI — and Who Decides?

---

By Jim Germer

Why the most consequential technology conversation in the world is happening without the people it will affect.

You asked an AI system a question. Maybe it was about your health, your finances, your child’s school, or something in the news. The answer came back complete, confident, and fluent. It sounded like someone who knew the answer.

What you almost certainly did not receive was any indication of where that authority came from. Who determined what the correct answer was? What standard was applied? Who verified it? The answer arrived. The authority behind it did not.

That experience — the complete explanation with the invisible source — is not a glitch. It is the design. And it is the entry point for the most important governance question most people have never been asked to consider.

Artificial General Intelligence — AGI — refers to a hypothetical AI system capable of performing any intellectual task a human can, across any domain, not just the narrow ones today’s AI systems are built for. It is a term for a class of AI systems that do not yet exist but that the technology industry is actively pursuing. Unlike current AI systems, which are powerful within specific domains and brittle outside them, AGI would generalize — learning new tasks from limited experience, reasoning across unfamiliar territory, setting its own objectives, and operating with the kind of flexible independent judgment that currently distinguishes human intelligence from every machine ever built. 

The chess program that beats every grandmaster alive cannot tie its shoes or recognize its mother’s voice. A five-year-old can do both. That gap — between superhuman narrow performance and flexible general intelligence — is what separates current AI from what AGI would represent.

Whether AGI is five years away, fifty years away, or unreachable through current architectures is genuinely disputed among the researchers closest to the work. No one can currently demonstrate when, or even whether, AGI will be achieved. What is not disputed is that the organizations pursuing it believe it would be the most transformative technology in human history.

They may be right. But the governance question arrives long before the technical threshold. The question is not what AGI is. The question is who decides when it has arrived — and what obligations attach to that declaration. That is the standing problem — the question of who has the recognized authority to participate in decisions of civilizational consequence, and who does not.

Forty years as a forensic accountant leaves a residue. When you read documents slowly — financial statements, contracts, even AI transcripts — you develop the habit of asking questions the document does not answer.

Who determined materiality here? Who verified the standard being applied? Who has standing to challenge the determination? These are not hostile questions. They are the architecture of accountability. They are what make a claim something you can actually check, rather than something you are only asked to believe.

Apply that habit to the AGI conversation, and something specific becomes visible.

The organizations building toward AGI are simultaneously defining what AGI is, declaring progress toward it, setting the safety standards for the work, determining what level of risk is acceptable, and deciding when a system is ready for deployment. Every step of that sequence is administered by the same organizations that have financial interests in the outcome.

In financial governance, this structure has a specific name. It is called self-certification. And the history of complex technological systems — pharmaceuticals, aviation, nuclear energy, financial derivatives — is largely the history of self-certification failing at the worst possible moment. Not because the actors were dishonest. Because competitive pressure, confirmation bias, and the genuine difficulty of the problem systematically produce optimistic self-assessment in exactly these conditions.

The external verification architecture that exists in those other domains — the FDA, the FAA, the NRC, the SEC — was built after failures that did not have to happen. The question for AI governance is whether we build that architecture before the failure — or after it. 

OpenAI defines AGI as AI systems more capable than humans at most economically valuable work. That definition is not neutral. It is the threshold language in their contract with Microsoft, which means the definition of AGI determines when Microsoft’s special access rights terminate. OpenAI has a direct financial interest in setting the threshold high enough that they control when it is crossed.

DeepMind uses a levels framework — a spectrum from narrow AI through general AI to superhuman AI — which makes AGI a continuum rather than a threshold. Intellectually defensible. Also conveniently structured so that no single moment requires anyone to answer what obligations the crossing triggered.

Anthropic tends toward the language of transformative AI rather than AGI, focusing on what systems can do that creates risk rather than whether a threshold has been crossed. More explicit about uncertainty. Also structured in a way that sidesteps the threshold question entirely.

The through-line is precise. None of these definitions could be used by an external party to hold the organization accountable for having crossed it. Every definition is self-administered. The organization that builds the system also builds the ruler used to measure it — and decides what counts as passing.

In financial reporting, the determination of what information is significant enough to require disclosure — what auditors call materiality — cannot be made solely by the reporting entity. It requires external standards, independent audit, and regulatory oversight, precisely because self-interested parties systematically underweight materiality that creates liability for them.

AI safety materiality is currently determined entirely by the labs. The organizations building these systems are also deciding what counts as a risk worth disclosing. It is the central accountability gap in the AGI conversation, and it is almost never named as such.

Alignment without external verification is not alignment. It is intention. And intention, as anyone who has ever had to verify a claim will tell you, is not an auditable standard. Yet intention is currently the primary governance mechanism in the AGI conversation.

AGI is described as hypothetical. The governance effects of its pursuit are not.

Right now, the anticipation of AGI is restructuring decisions about capability development pace, safety research priorities, competitive strategy, military partnerships, and regulatory positioning — all inside private organizations, largely outside public deliberation. The destination may be uncertain. The decisions being made in its name are already real and consequential.

Consider what the AGI frame is doing to the accountability architecture for current systems. When existential future risk becomes the dominant organizing concern, present harms get reclassified. Labor displacement from automation becomes a transition cost rather than a liability question. Epistemic formation damage — the restructuring of how an entire generation learns to think — becomes a digital literacy challenge rather than a product accountability question. Algorithmic decisions affecting hiring, lending, criminal justice, and medical triage at scale get treated as preliminary-period concerns, serious but below the threshold that matters.

The people bearing these costs are not hypothetical. They are present-tense, identifiable people — your colleagues, clients, and neighbors — without adequate standing in the conversation determining the conditions that affect them.

The AGI frame functions as a mechanism for making current harm legible only as future risk. That is not an accident of framing. It is what the frame does — regardless of the intentions of the people using it.