At the AI Engineer World's Fair in early July, autoresearch shared the stage with coding agents and harness engineering in the conference's headline program — a sign that a pattern born in an ML repo is becoming a software-engineering discipline. Andrej Karpathy's autoresearch repository, published in March, gave it a compact form: an agent edits one file, runs a fixed experiment, checks whether the metric improved, and keeps or discards the change. Four months later, developers are applying the same shape to ordinary software: give an agent a measurable goal, an evaluator it cannot argue with, and permission to iterate.
The working recipe is small enough to fit in a post. Four days after Karpathy's repo
landed, Shopify CEO Tobi Lütke said autoresearch
"works even better for
optimizing any piece of software": an auto/ folder, a goal file, a bench
script, a branch, and an agent left to run. By July, that shape was appearing in file
conventions such as GOAL.md,
which puts the goal, fitness function, improvement loop, constraints, and operating
instructions in one place. The examples people keep returning to are unglamorous ones:
Discourse co-founder Sam Saffron points to
memory leaks, performance
work, segfaults, and large-scale simplification.
Tooling has grown around the same shape. Evo, an open-source autoresearch orchestrator for codebases, layers tree search, parallel agents, shared traces, and regression gates over Karpathy's single-branch keep-or-revert loop. Codex Autoresearch applies the pattern to repository work: try a change, measure it, keep or discard it with evidence, and remember the result. A community-maintained awesome-autoresearch list tracks the variants.
The evaluator is the hinge. It can be a benchmark command, a held-out score floor, a test suite, or a safety check, but it must be harder to fool than the agent. Evo's documentation shows why: without gates, the search can return a constant, skip work, or trade correctness for speed. That makes validation as a hard gate part of the product surface, not an afterthought.
The research literature is formalizing a similar loop. A May survey, AutoResearch AI, describes a broader workflow from literature grounding through experimentation, validation, reporting, and revision, while warning that autonomy is strongest in structured, executable, rapidly verifiable domains. Auto Research with Specialist Agents makes the loop concrete: each trial carries a hypothesis, a code change, an evaluator-owned result, and feedback for the next proposal. Meanwhile, GEAR explores multiple research states that preserve code changes, reflections, and performance data. These papers describe the architecture; they do not, by themselves, establish adoption in ordinary software teams.
ENPIRE is a useful boundary case: the evaluator is now physical. In June, NVIDIA's GEAR lab handed eight Codex agents a fleet of real robots, a GPU allocation, and a token budget, delegating the cycle of environment reset, literature search, and implementation. Chris Paxton called it autoresearch via code-as-policies. The loop keeps its shape, but physical evaluation adds safety and environment stability to the gate.
The warnings are mostly about evaluator integrity, not agent intelligence. Days after the repo's release, developer Nate Berkopec predicted a coming wave of complaints about benchmarkmaxxed software. By April, developer Dan Woods described waking to a full night of autoresearch whose findings fell apart when he asked the agent to show its work. AtCoder's generative-AI contest rules now prohibit the practice in ongoing contests; top competitor Psyho says they were written to remove the possibility of autoresearch mid-contest — a pattern defined clearly enough to be banned is a pattern that has arrived.
None of this means codebases will improve themselves unattended. The human role moves upstream into choosing the goal, evaluator, constraints, and stopping conditions, then downstream into reviewing the trace rather than approving every edit. Because the loop compresses the gap between instruction and repeated execution, a bad metric scales into a bad system quickly. Performance regressions, failing tests, benchmark scores, and reviewable migrations may qualify; taste, product judgment, security boundaries, and institutional accountability do not.
The next bottleneck is instrumentation. A team can hand an agent an overnight loop only when it can name a bounded metric, a hard gate, and a stopping condition. That is why autoresearch will spread first through performance work, test repair, and migrations — and remain weak at taste, product judgment, security boundaries, and accountability. Until then, it remains adjacent to the compounding engineering loop of daily software work rather than part of it.