Your AI Is Following Orders You Forgot You Gave
For weeks I was correcting the same problems in every draft my system handed me. Not new problems, the same ones, week after week. I was rewriting entire sections that lacked quality but were in the area of what I wanted to say. That phrase is the most precise description I have of degraded AI output. Nothing is wrong enough to call broken. Everything is close, but not quite ready for publishing. I would sit down with a draft, recognize the shape of my own argument in it, and then spend the evening dragging it the last forty percent of the way home.
The time stung more than the quality did. I built this system so that AI could help me bring my thoughts and analysis to life. I write about that process every week, because Neural Gains Weekly exists to make AI more accessible by building in public. A production system that eats my evenings in cleanup is not just inconvenient. It is a sign I had something new to learn, and learning in the open is the entire premise of this newsletter.
When the quality slipped, I did what most builders would do and reached for more rules. I tightened instructions, added checks, wrote scans for the exact failures I kept seeing. Every one of those edits felt responsible while making the output a little worse. Through all of it I blamed the model, and it never occurred to me that the rules I had written might be the thing dragging the drafts down.
Then the labs put it in writing. I caught headlines about OpenAI telling developers that the newest models need a different kind of prompting to generate great output, that the guidance written for last year's models was now working against this year's. I read that with the specific discomfort of recognizing your own house in a news story. So I opened my terminal assistant, pointed it at the current published guidance from the labs, and asked it to audit my whole skills system against what the model makers themselves now recommend.
Correct on the Day You Wrote It
What came back reorganized how I think about every instruction I have ever written, and it fits in two words. Constraint and compensation. Some of your rules are constraints. They exist because of something true about you, your voice, your privacy lines, the standards your audience expects, and they hold no matter which model runs underneath them. The rest are compensation. You wrote them to stop a weaker model from doing something you did not want. Verify your work before delivering it. Double-check the numbers. Ask me before you make any changes. Each one was correct on the day you wrote it, because on that day the model genuinely needed telling.
A constraint holds through any upgrade, because it was never about the model in the first place. Compensation ages, and a rule that has outlived its reason does not just sit there harmlessly. An instruction telling a model to do something it already does by default stacks on top of the built-in behavior. The result is a model that overdoes the very thing you asked for, re-checking work that was already checked and burning time and tokens to do it. This is not my theory. Anthropic's prompting guide for Claude Opus 5 tells developers to delete verification instructions outright, because the model already verifies its work and the carried-over rule causes over-verification with no gain in quality. The same guide goes further, telling you that if you carried your effort settings over from an older model, you should re-run the sweep against your own evaluations instead of trusting the numbers that worked before. The company selling the model is telling you your instructions are hurting it.
In the best case, a stale rule fails loudly. On Claude Opus 5, a carried-over setting that turns thinking off while asking for the highest effort levels now comes back as an error, and an error gets fixed the same week because it refuses to be ignored. The dangerous rules are the ones that keep working. They still run, the output still arrives, and nothing anywhere flags that an instruction has started costing you quality instead of protecting it. A rule that looks like diligence can sit in a production file for months doing quiet damage, and mine did.
What I Deleted, and What I Defended
The agents in my terminal session came back with proposed changes to nearly every skill file in my production system. I reviewed them the way I would review any contractor's work, one proposal at a time, accepting what held up and pushing the approved changes into my repository. My article workflow came out the other side with four stages collapsed into three, and one finding I still think about.
Back when models could not hold a long piece together, I wrote a rule into my drafting chain instructing the writer to deliver a partial draft and leave the rest to me. That was a deliberate choice and it was smart at the time. On a frontier model it meant I had automated my own rework. I was paying for a system that was required to hand me unfinished work, then spending my evenings finishing it and wondering why the drafts felt thin. Around that rule sat the smaller ones, blocklists of banned punctuation and phrases standing in for judgment I did not yet trust, and duplicate scans running at multiple stages that doubled cost without doubling quality. Most of the blocklists went back to judgment. A few stayed, because they turned out to be about my voice rather than a dead model's bad habits.
The audit got one round wrong, and the mistake taught me as much as the findings did. The agents cut too far, stripping structure requirements out of my brief that I had written on purpose, and the drafts that followed had more leeway than I was willing to hand any model. So I pushed back and wrote the guardrails on paragraphs and structure back in. That disagreement turned out to be the most useful moment of the rebuild. It forced me to say out loud why each rule existed, and a rule you can defend out loud is a constraint.
When the rebuild was done, my system was smaller than when it started. Nearly everything that changed was a deletion. What went back in, I could defend.
Green Dashboards, Worse Answers
I caught this because my newsletter is the one output I read line by line. Every other skill I have built was drifting the same way, running unwatched, producing work slightly worse than it should have been in ways nobody would ever flag.
The system's literature has a name for this. Version drift is what happens when a provider updates or retunes a model and previously stable workflows change their output format, reasoning style, or tool-call patterns. Same code, same prompts, different behavior. A researcher at Microsoft Security Research, cataloguing fifteen ways these systems fail in production, describes the problem sitting underneath. Silent regression is a model update that degrades behavior while every dashboard metric stays green. Teams instrument latency and error rates, and almost nobody instruments whether the answers got worse.
When researchers compared two versions of GPT-4 a few months apart in 2023, instruction-following had collapsed, from honoring a formatting instruction in 99.5 percent of queries to almost none. A study published in April 2026 found the same shape in two families of smaller open models. Both upgrades posted aggregate benchmark gains. Yet among the test items where change could be measured, three in ten reliably got worse in one family and four in ten in the other, and the losses clustered by subject, physics in one, law in the other. The headline gain was only what was left after the wins and losses cancelled out, which means a team upgrading on that number alone would never see what it lost.
I have spent my career around regulated systems, where the assumption is that a validated process stays validated until somebody changes it. Model upgrades break that assumption from the outside, on a schedule the vendors publish and you do not control. I had evaluation. I did not have governance. Knowing the difference professionally did not stop me from missing it in my own shop.
So here is the work, and it is smaller than this article makes it feel. Open whatever holds your standing instructions, the custom instructions, the saved prompts, the project files, and read each rule against one question. Can I point to a session where this model actually did the thing this rule prevents? If yes, it is a constraint and it stays. If no, it is compensation for a model you are no longer running, and it goes. Somewhere in that file is an instruction telling the model to do something it now does by default, which means you have been paying twice for a worse result. Delete that one first, then put a date on the next review, because this will happen again on a schedule somebody else sets.
The rebuild is done, and every skill that came out of it is under evaluation now. Some results I will see quickly. Most I never will, because the improvement is invisible by definition. My instinct was backwards, and I knew it before any results came in. When the output got worse, I tried to add my way out. The answer was subtraction. Build for the model you have now, not the one you had when you wrote the rule.