Thermodynamic Programming: What Happens When 10 Agents Share a Branch?
Months ago, we published a Verilog simulator built by AI agents over 43 days.
Early on in a software project's life, there is a lot of code to write, and a lot of architectural decisions are still in flux. It is common to see large PRs and frequent refactors. In the Verilog Simulator Project, we decided to take this to the extreme and have all 10+ agents working in parallel on the same codebase. No PRs, no separate git worktrees. Agents moved fast and constantly bumped into each other, but contrary to ordinary best practices in software engineering, it worked. Velocity held in a way it would not have with a human team, where tracking what changed while you were writing would overwhelm any parallelism gains. The shared branch is only viable when the agents resolving moving conflicts are more capable than the conflicts themselves. We began to notice this same structure in other places, notably how neural networks train, how collaborative documents evolve, and any system that has to balance exploration against convergence.
The standard response to parallelism in software is to define clean API boundaries up front and divide work across them. This reduces coordination overhead, but carries a hidden cost: until the ambiguity in the problem space is sufficiently resolved, the API itself will need to be constantly refactored. Every time the API changes, the teams or agents working against it have to adapt. Fixed boundaries are a coordination tool that works well in a stable system and creates friction in a rapidly evolving one.
The shared branch, high-temp method sidesteps this. Parallelism scales with agent count, and when one agent refactors code below another, the second agent adapts and moves on. At the end, the codebase is consistent with no need for merges. This is only possible with highly capable agents. We can think of entropy in a codebase as technical debt; things like inconsistent naming, duplicated utilities, and unclear structure are all the friction costs of unsynchronized parallel work. Some codebases can absorb more of this than others. For example, a codebase with independent components and robust test coverage has higher heat capacity and can run hotter before the system destabilizes, but all systems eventually require annealing.
Annealing means reducing temperature: moving agents to isolated branches, requiring review before changes land, and resolving accumulated inconsistencies. At this stage, agents are solving difficult bugs that may take hours to trace -- work that requires isolation. A code reviewer acts as Maxwell's demon, selectively letting good changes through and rejecting bad ones, locally decreasing entropy at the cost of reviewer energy. The result, over time, is a codebase that has settled from a chaotic initial state into something coherent and stable.
The Hogwild algorithm trained neural networks asynchronously across a compute cluster: each server took a snapshot of the model weights, performed some training, and sent back a weight delta to a shared server. This is high-temperature training (maximum parallelism, no coordination between servers), which is not so different from batch gradient descent, where you apply the same weights to many data points and average the updates. What’s interesting is that Google eventually moved from asynchronous to synchronous training, where all servers step forward in lockstep applying their delta patches at the same time. Google researchers found that achieving the last 1-2% of accuracy (e.g. getting from 95% to 97%) was harder in async mode. For them, this could mean the difference between SOTA and not getting a paper accepted. This explanation looks familiar: in high-temperature mode, gradient updates conflict -- two servers compute updates from the same starting weights, and the aggregate drifts in contradictory directions. When the model is far from convergence, this is acceptable because the loss landscape is rough and the model has room to move. As the model approaches its target, conflicting updates start canceling each other out and progress stalls. It appears that synchronous training is the annealing step: it imposes coordination, reduces parallelism, and lets the system converge.
The same structure appears in collaborative writing. In the early drafting phase, a shared document runs hot: multiple contributors add ideas simultaneously, structure shifts, arguments get reworked. This phase is productive precisely because coordination cost is low -- contributors do not need approval before adding a sentence. The entropy of conflicting edits is a feature: it produces more raw material faster. As the document approaches its final form, this mode becomes counterproductive. A late-stage edit to a near-finished piece risks breaking the argument, introducing inconsistency, or undoing a deliberate structural choice. At that point, commenting and approval workflows serve the same role as annealing in code: they slow the rate of change, let the document settle, and preserve the coherence that accumulated parallelism produces.
In the future, will we be able to run codebases even hotter? There are many examples from physics, where finding a way to run a system at a high operating temperature has been key to unlocking unprecedented efficiency, such as jet engines. When we first started using Claude and Codex, they were unable to handle a shared worktree, and would constantly try to run git reset to remove all the “unexpected changes”. As we train agents with RL to work in this kind of environment, they learn to be more resilient when operating at this energy level.
Right now the Verilog Simulator project is still running at too high a temperature for a (human) team to collaborate. Once we start taking PRs on GitHub, it'll be a sign that we have moved close to its final crystalline state. But in the near future, once we start trusting AI to review PRs, we'll have unlocked multi-player vibe coding, where teams of engineers can all operate at a relatively high temperature, even in the final phases of a project.
