Being in the Loop Is Not Enough

Every post I publish through Ghost, the platform this site runs on, appears in the archive automatically. I wanted the same thing to happen with episodes of Explore AI Out Loud. Those publish on YouTube, so I needed a connection that would bring each episode into the site’s archive when it went live, without me adding it manually.

I gave the task to Codex running on GPT-6 Astra. Using its computer-use capability, it could open a browser, read the screen, click, and type across the services involved. I watched it configure the connection between YouTube and Ghost and test it using a video I already had loaded in YouTube.

Before running a live test from beginning to end, Codex asked for my approval. I approved it and went back to other work. When I returned, Codex reported that the test was finished, the temporary changes had been rolled back, and the results had been verified. Its report said the test video was back to Private and the website had been restored to its pre-test state.

Along with that report, Codex had produced receipts showing what it had checked and completed the project records documenting the work. It had carried the task through testing, cleanup, and verification, then written the documentation I would normally review to decide whether the job was complete. All while I worked on other tasks.

I Knew What to Check

I opened the Google console first and checked that the new connections were there. Then I went to YouTube and confirmed the test video was back to Private. On the website, I checked the archive’s layout and looked for anything left over from the test. I knew that page from building it, and it looked the way it had before the run.

I also read the reasoning shown in Codex. It was less detailed than what I had seen from past models, but I could follow the steps and connect them to the results I was checking. Everything lined up. Nothing I found contradicted the report.

Looking back, I realized how much that review depended on what I already knew. I knew which connections should have been added, which video setting should have been restored, and what the archive should look like. Without that context, I could have opened all three services and still missed a change that should not have been there.

Codex had given me a complete account of the job. My familiarity with those systems gave me something to compare it against. That experience was on my mind as I followed the argument about whether Astra marked the arrival of artificial general intelligence, or AGI.

How Much Can the Reasoning Tell Us?

After Astra’s release, Nvidia’s Jensen Huang declared that “AGI has arrived.” The ARC Prize team, whose benchmark featured in OpenAI’s announcement, celebrated the progress but explicitly declined to call it AGI. Even agreement that something significant had happened did not produce agreement about what to call it.

I had already argued in Volume 19 that the definition mattered less than the work these systems could do. Watching Astra operate my computer made that argument feel much more immediate. Seeing it move between services, configure the connection, and carry the task through testing was amazing. This was work I needed finished for the podcast, and it was getting done while I handled something else. I could appreciate how remarkable that was without knowing whether AGI was the right name for it.

But the experience also left me thinking about how I would audit increasingly capable systems. OpenAI’s system card, which documents its testing and limitations, gave that question substance. Researchers tested how well another AI could judge aspects of Astra’s behavior using its written reasoning, records of its actions and outputs, or both. Compared with the earlier GPT-5.6 Sol, Astra was harder for that AI to assess when reasoning was available, even alongside its actions. When the reviewer saw only actions and outputs, its scores were higher for Astra. OpenAI cautioned that some of those gains did not reflect meaningful improvements in monitoring.

Much of the reasoning gap came from shorter traces that left monitors with less useful evidence. Separate tests deliberately encouraged monitoring evasion. Those results should not be read as ordinary behavior. OpenAI also found that Astra generally respected safety and security restrictions more reliably than Sol. Reasoning remained useful for monitoring, despite the decline. The findings support developing additional ways to audit these systems.

My review was much simpler than those experiments. I was checking whether a particular job had been completed and cleaned up correctly. Reading Codex’s account helped me follow what happened, but I could also open the services and inspect the results. The research did not prove that my checks were sufficient. It made me appreciate having something beyond the model’s explanation to work with. Using that evidence depended on knowing what I was looking at.

What I Am Still Responsible For

“Human in the loop” feels like an outdated description of what this work requires. You need auditors in the loop who know what to look for. I could read every line of a completion report and still miss a problem if I did not understand what the changes meant. For this job, I had enough familiarity with the systems to investigate beyond what Codex told me. I would not assume that familiarity carries over to every task I could now ask it to perform.

That is where computer use becomes both exciting and demanding. I can hand over more of the execution and get on with other work. But when I come back, I still need a way to judge the result. The more a system can complete on its own, the more deliberate I need to be about which parts I am equipped to review.

I think this helps explain why putting agents to work inside large organizations can be difficult. In my experience, legacy systems often depend on undocumented manual checks and workflows that differ from the written procedure. The people doing the work know those differences, even when the documentation does not capture them. My concern is that an agent could follow the documented process and produce a convincing report while missing something those people would know to question.

As I write this, Astra is running the final editing workflow for Episode 1. I am continuing to use it because what I have experienced is remarkable, and I want to understand how far it can take the work. I am also trying to understand what that leaves me responsible for.