That is a substantial amount of activity around a project I work on with AI agents. It includes product code, tools, tests, documentation, experiments, and generated material. It also includes merges and multiple author identities. I am describing the history of the repository I maintain, not claiming that I personally typed every change.
The numbers are useful context. They are a poor definition of success.
Git gives me a record of changes and decisions. I can see the tooling expand, trace an architectural choice, or identify when a new validation path appeared. I can compare a fixed pair of endpoints and reproduce the result.
It cannot tell me how many pull requests were reviewed without a separate source for PR data. A directory full of tests does not tell me how many passed. A large line count does not distinguish a valuable system from generated output or expensive rework.
Even the commit count depends on the query. Traversing every local branch can include work outside the history of the endpoint I intend to describe. A credible number needs a scope as well as a value.
I have included the snapshot and counting method so the terms are explicit. There is no defensible conversion from these counts to a claim that the whole project became ten times more productive.
The more interesting history is the development of the feedback loop.
Early work added a browser editor toolchain and automated runtime routes. Subsequent work added metrics, visual captures, shared schemas, and targeted checks. In September, I tightened the operating rules around focused smoke validation and reusing passing evidence while the relevant inputs stayed unchanged.
Those changes gave me more ways to examine the product and more precise tasks to give agents. They support iteration. They do not automatically prove that every resulting feature is good.
The next question is how reliably this system turns a selected problem into an accepted result. That is where I would put the measurement effort for an engineering organization.
I would start with a small set of measures tied to delivery:
| Measure | Working definition | What it helps reveal |
|---|---|---|
| Time to accepted result | Task selection to a result that meets agreed acceptance criteria | Whether the complete workflow is getting faster |
| Review waiting time | Time a change waits for an accountable reviewer | Whether generation is outrunning review |
| Rework | Changes reopened or substantially redone after acceptance | Whether apparent speed creates downstream work |
| Escaped defects | Relevant defects found after release, by severity | Whether faster delivery preserves reliability |
| Verification effort | Human and machine effort needed to establish acceptance | Whether tooling makes confidence cheaper |
| Customer or product outcome | The behavior or business result the work was intended to improve | Whether the organization chose useful work |
These are proposed measures, not results from an instrumented study of my project or Samsara. Their definitions would need to be consistent before comparing periods or teams.
I would compare similar types of work and retain the cost of review, testing, and rework. Timing only the implementation step can make a workflow look faster while pushing its cost onto someone else.
METR’s early-2025 randomized study is a useful challenge to intuition. In that study, 16 experienced open-source developers handled 246 tasks in repositories they knew. Tasks with AI access took 19% longer on average, even though participants believed the tools had helped them work faster.
That finding concerns a specific set of developers, tasks, and early-2025 tools. It is not a verdict on my workflow or every later agent system. It is a reason to measure the outcome rather than rely on the feeling of acceleration.
The 2025 DORA research approaches a different scale. Its survey-based findings describe AI as amplifying the organization around it, with increased delivery throughput alongside continuing instability. That is an association from organizational research, not a controlled estimate of what one tool will do for one team.
Both perspectives make me more interested in the surrounding system: ownership, platforms, feedback, and the ability to absorb change.
My ambition is to increase the useful capacity of the teams and systems I lead. At Samsara, that means thinking across devices, firmware, testing, automation, data, and backend services. In my independent work, it means learning quickly enough to make a better product with the time I have.
Those settings have different constraints. A successful game-development experiment does not establish a fleet-wide productivity result. It can still produce a practice worth testing in another setting.
The question I would put in front of an executive team is specific: can we deliver a chosen class of valuable changes sooner, with acceptable cost and reliability, and sustain the improvement?
Commits help me investigate how the work happened. The answer to that question tells me whether the investment worked.
See the complete AI engineering series for the practices behind these numbers.
]]>I assigned agents to individual levels in my game. Each could change its own files. I could rebuild, visit a level, decide what needed work, and send that feedback to the agent responsible for it. Several pieces improved at the same time without much coordination.
Then a machine update took out my open sessions. When I restarted, I reorganized the work around themes: tutorials, controls, bug fixes, UI improvements, and other changes that sounded like sensible projects.
Those themes crossed the same codebase. Eight agents could now make different decisions about shared systems. Changes accumulated into large commits. Eventually, levels stopped loading. Finding the problem took hours, and reverting was difficult because the changes were tangled together.
I had made it easy to start work and hard to recover from it.
The rule I use now is one writer per overlapping scope. That can mean a file, a subsystem, or a contract spanning runtime code and an editor. The boundary needs to reflect what can actually change together.
“You own the UI” is usually too broad. “Update this shared list component and its affected callers; another task is investigating layout without editing” gives me a usable division of work.
Read-only work is easier to parallelize. An agent can inspect a subsystem while another examines failures or prepares a review. When two implementation tasks really are independent, both can run. When they meet at a shared contract, I make the dependency explicit and serialize the overlapping edits.
Separate branches help isolate files, but they cannot reconcile incompatible design assumptions for me. Ownership still matters at integration time.
I use a small task brief:
1
2
3
4
5
6
Outcome: What should be observably different?
Ownership: Which files or subsystem can this task change?
Constraints: Which existing contracts must it preserve?
Evidence: Which check will demonstrate the intended result?
Escalation: Which discovery changes the scope or needs a decision?
Handoff: What changed, what passed, and what remains uncertain?
The evidence line is important. If I cannot describe how I will judge the result, I probably have more thinking to do before delegating implementation.
This brief also lets an agent act without asking me about every reversible choice. I am trying to clarify the decisions that matter and remove friction from the rest. Requiring approval for every small step would throw away much of the benefit.
Small, coherent changes are easier to inspect and undo. They preserve a connection between the problem, the decision, and the evidence. A large diff containing several unrelated improvements breaks that connection, even when every individual edit seems reasonable.
I also keep the current implementation and the latest handoff ahead of old plans. Plans describe an intended future. In a fast-moving project, some of them become archaeology surprisingly quickly. An agent that confidently follows an obsolete design can create more work than an agent that stops to identify the current owner.
The durable information belongs in the repository: sources of truth, build commands, known constraints, and short handoffs. Chat is useful working memory. It is a poor place to store the only explanation of why a decision was made.
There is a second failure mode: adding so much verification and coordination that a simple fix becomes a procession of agents and reports.
In September, I changed the repository’s operating rules to default to quick, focused smoke checks. A routine change should receive the smallest meaningful validation, and passing evidence should be reused while the relevant inputs remain unchanged. Broader sweeps have a place, especially for shared behavior and release risk. They should have a reason.
A documentation correction does not need a game build. A small tool change should not automatically trigger a full gameplay campaign. A shared save-system change deserves much more scrutiny. The risk determines the check.
I apply the same reasoning to delegation. One implementation owner is often enough. A reviewer or investigator earns its place when it provides a distinct kind of evidence, not when it repeats the author’s conclusion.
I judge a guardrail by the decision it makes easier. Does it prevent conflicting edits? Catch a failure close to its cause? Make a result reviewable? Let the next session continue without guessing?
If it does none of those things, it may be ceremony.
The rules I kept after that early failure have made parallel work more manageable. They let me grant autonomy with a clearer understanding of what I will get back. That is the kind of process I want: enough structure that I can move quickly and still know where I am.
Related: What an AI agent has to prove before I call it done.
]]>That distinction has become the center of how I build. I run multiple AI sessions as part of my daily work on All Is Vanity, a game I have been bringing toward a commercial release. I use it as a place to exercise the whole engineering loop: choose a problem, build something, test it, look at it, change my mind, and try again.
I can delegate implementation, tool changes, and investigations in parallel. That also makes weak direction expensive very quickly.
If I ask for six improvements without deciding which problem matters, I can get six polished answers to questions I should never have asked. Someone still has to choose the target, recognize a good result, and decide when another iteration is worth doing. That is my job.
“Make this better” leaves almost every important decision unresolved. “Add these three features” can be just as bad if the features are guesses about a problem I have not understood.
I get better results when I describe what someone should experience. For example: the player should recognize an incoming threat before taking damage, understand what happened afterward, and be able to respond without studying a manual. Those are observable goals. Increasing a glow effect or adding another animation is an implementation option.
Once the goal is clear, I can ask an agent to inspect the current behavior, identify where the signal gets lost, and propose a bounded change. I can also reject a technically correct result because it makes the screen busier without making the action clearer.
That is a useful rejection. It tells us something about the product. A vague request followed by “still doesn’t feel right” tells us much less.
My workflow has developed three complementary parts. I set the intended outcome and the constraints. Agents explore and implement within a defined scope. Tools make the result cheap enough to inspect that I can keep steering.
The tools matter as much as the conversation. I have built editors, preview surfaces, repeatable scenarios, runtime probes, and capture workflows around the game. They let me get back to a specific state and compare a change instead of spending the next session reconstructing the previous one.
This is where the work compounds. A better iteration surface helps with the current change and the next hundred changes. Another long prompt usually helps with one conversation.
I have already tried broad mandates with lots of files changing at once. The guardrails essay describes the failure that made me reconsider that approach.
My current rules are more specific. One owner controls an overlapping write scope. Read-only investigation can happen in parallel. Tasks have a defined outcome and a verification path. Changes to shared contracts require coordination. A second agent needs a reason to exist beyond the fact that another session is available.
These rules make it easier to use the speed. I spend less attention reconciling competing assumptions and more attention judging whether the product is improving.
There is still an uncomfortable part: deciding not to build something. AI lowers the cost of trying ideas, but maintenance, interaction complexity, and user attention remain expensive. A feature that is cheap to generate can still be costly to own.
At Samsara, my scope spans Safety and Telematics, including firmware, quality engineering, automation, and services connecting devices to the backend. The constraints are different from a game: physical hardware, constrained compute, system reliability, and teams that need to make decisions together.
But the leadership question is familiar. How do I give people a shared target and enough feedback to make good decisions without routing every detail through me?
My hands-on agent work gives me a fast way to examine that question. I can see what happens when ownership is fuzzy, when testing is too expensive, when the wrong metric becomes the goal, or when a tool makes an entire class of work easier. These are lessons I bring to organizational design; the game is not a benchmark for Samsara’s productivity.
I want substantially more capacity from the teams and systems I lead. The way to pursue it is to improve the path from a useful idea to a verified result, then measure what actually changed.
AI gives me more opportunities to act. Direction determines which actions are worth taking. The feedback loop tells me whether we are getting anywhere.
This is the opening essay in my series on building with AI. The next piece covers the guardrails that make parallel work useful.
]]>My scope has changed more than once across my career, including at Samsara. Responsibilities expand, teams move, and relationships need to be rebuilt. The product still needs attention while people are working out what the new arrangement means.
I think keeping teams productive and morale intact through those transitions deserves serious weight in how we evaluate leaders.
People can usually work with a difficult constraint if they understand it. Ambiguity is harder when it affects who decides, which work matters, or how someone can succeed in a role.
During a transition, I want to clarify what has changed and what has not. Which commitments remain? Who owns a decision now? What is still unresolved? When should someone expect an answer?
I cannot remove every uncertainty. I can avoid disguising it as confidence, and I can keep unresolved questions from becoming everyone’s private guessing exercise.
That requires repeated communication. A decision explained once in a meeting is not necessarily a shared understanding across the people doing the work.
An organization chart describes formal responsibility. It does not capture all the relationships that make a product work: the engineer who understands a device behavior, the reviewer who knows a difficult history, or the manager who helps two disciplines make a decision together.
I want those connections to survive the transition where they remain useful. New boundaries should make ownership clearer without making collaboration unnecessarily difficult.
This is particularly important in systems that cross hardware and software. Moving a responsibility on paper does not instantly transfer the context needed to exercise it well.
The work includes handoffs, shared priorities, and enough overlap for people to develop judgment in the new arrangement.
Teams need visible progress on the product as well as clarity about the organization. If everything becomes an internal coordination exercise, people can lose the connection between their work and the reason they joined.
I try to keep a manageable set of concrete outcomes in view. That gives the team something to complete together while the longer-term structure settles.
It also creates useful feedback. If the new arrangement makes an important decision harder, I want to notice and adjust it. Organizational design should be judged partly by what it enables the team to do.
I write a lot about directing AI agents. There are useful parallels around clarity and ownership, but the analogy has limits.
People bring relationships, ambitions, concerns, and developing expertise. A good organizational change has to account for those things. Improving delivery at the expense of trust or growth can damage the system I am trying to improve.
I want managers to have room to lead, engineers to understand how their work matters, and people to see a credible path for themselves as the scope changes.
That responsibility remains human. Tools can increase execution capacity. Keeping an organization able and willing to use that capacity well is leadership work.
]]>More changes reach the queue. More combinations need investigation. The people validating the product have to reconstruct what each change intended, which assumptions it made, and how to exercise the relevant behavior.
If I respond only by asking that group to move faster, I have missed the design problem.
At Samsara, my responsibilities include quality engineering and automation alongside the firmware and systems work. I see that combination as an opportunity to improve the development loop itself.
I want a change to arrive with a clear statement of the behavior it affects and an appropriate way to examine it. That might involve an isolated check, a simulator, an emulator, a device, or an interaction with backend services.
Each environment supplies a different kind of evidence. The goal is to find important failures early without pretending that one environment answers every question.
For device software, actual hardware matters. It is also a resource that requires thoughtful automation, reliable setup, and ownership. The useful engineering work includes making the environment repeatable enough that a failure tells us something about the change.
An unreliable setup can teach teams to distrust the signal. That is an expensive form of noise.
An agent can help create scenarios and exercise a system. It can also generate tests that reproduce the same misunderstanding as its implementation.
I want the intended behavior and consequential failure cases to remain explicit. The question is not only whether a test passes. It is whether it would fail for the reason we care about.
This is why the right guardrails can support speed. A clear acceptance boundary lets implementation proceed with fewer interruptions. A useful failure signal brings the work back while the context is still available.
Human review should concentrate on decisions where judgment is necessary. Automation should make the repetitive evidence easier to obtain. Neither becomes better merely by being applied everywhere in the same way.
Quality engineers often see patterns that individual feature teams encounter only occasionally: difficult setup, recurring ambiguity, fragile transitions, and failures that escape local checks.
I want those observations to shape the development interfaces and the tooling backlog. A recurring testing burden may be evidence that the product is hard to observe or that ownership is unclear.
Fixing that structure can help more than adding another manual step at the end.
There will still be independent validation and release judgment. Bringing quality into the loop does not eliminate those responsibilities. It gives them a better starting point.
The measures I care about include how quickly a change reaches useful feedback, how often a failure can be reproduced, and how much rework appears after acceptance. I want to understand the cost carried by the people reviewing and validating the result, not just the time spent producing it.
My independent game work gives me a smaller place to practice the same principle. Direct routes, harnesses, and repeatable captures make it easier to judge an agent’s work while it is still a small change.
In both settings, the objective is a development system that helps people discover the truth about a change sooner. That is a more durable source of speed than making the final queue work harder.
]]>That distinction matters when agents can produce implementation and a convincing explanation in the same session. I need a way to evaluate the result independently of the explanation.
For my game, I have built that around targeted runtime checks. The project has harnesses for gameplay, UI, abilities, procedural generation, effects, and other systems. Those tools are part of the development interface: an agent can use them to observe behavior and return evidence I can inspect.
Here is how I expect that loop to work.
For a bug, the preferred sequence is to reproduce it in a test, observe the test fail, change the implementation, and observe it pass. The failing run matters. Without it, I may have a test that never exercised the problem.
Suppose a confirmation button looks enabled but does nothing. A source-text check that finds the button label is not useful evidence. Neither is a screenshot showing that the button exists. The check needs to reach the relevant screen, trigger the control, and observe the expected state change.
That example sounds obvious. It is also the kind of distinction an agent can miss when the task is framed as “add a test” instead of “demonstrate this behavior.”
I remain responsible for the scenario and the assertion. A test generated from the same misunderstanding as the implementation can agree with it perfectly.
The development loop uses a browser target for quick feedback, while Windows is the release target. For compiled game changes, I use the full build pipeline and verify that the local server is serving the intended output.
This catches an easy source of false confidence: testing an older bundle. It also prevents a shortcut compile from bypassing framework setup that the runtime expects.
Once the build identity is established, the harness must reach the intended state. A process that exits without errors after opening the wrong scene has not tested the change. The route, relevant configuration, readiness signal, and resulting action all belong in the evidence.
A useful completion report is short and specific:
1
2
3
4
5
6
7
Change: What behavior changed?
Build: Which output did the runtime load?
Scenario: Which route and deterministic inputs were used?
Check: What action ran, and what observable result passed?
Errors: Were there new console or network failures?
Limits: What was not exercised?
Cleanup: Were test-owned pages and processes closed?
I do not need a wall of logs pasted into every conversation. I need the conclusion and a path to the supporting output when I want to investigate.
This also improves handoffs. Another session can continue from a known state instead of repeating setup or accepting “tested” as an unexplained fact.
More tests are useful when they add information. Repeating a passing check without changing its inputs may only add time.
A tool-only change gets the affected tool interaction. A game change gets a build and a relevant runtime smoke. A shared contract or progression change may need wider coverage because its failure can travel. Changes involving native behavior need exercise on the actual target; browser success cannot establish Windows parity.
Visual and gameplay judgment remain separate. A harness can tell me that an effect triggered, stayed within a limit, and cleaned itself up. I still need to decide whether it communicates the right thing. An automated player finishing a sequence does not establish that a human will enjoy it.
This is why I resist both extremes: accepting a compile as proof of everything, and treating an exhaustive sweep as the default price of every edit.
Browsers, local servers, traces, and test processes consume real resources. If a harness leaves them behind, repeated testing can make the workstation slower and the evidence less reliable. Cleanup is part of the test contract.
So is choosing a cheap route back to the behavior. If checking a small change requires manually navigating five screens every time, that is a tooling problem worth considering. The easier it is to inspect the result, the more often I can exercise judgment while the change is still small.
Simon Willison’s practical guide to coding with LLMs makes testing a responsibility of the person using the tool. My version of that principle is to make the proof part of the assignment.
When an agent returns, I want to know what changed in the running system. That gives me something more useful than confidence: a result I can check.
]]>A prototype can answer a question without being a good foundation. It may choose a convenient data path, ignore lifecycle details, or build its own version of a shared component. Those choices can be reasonable for an experiment and expensive in production.
I want to preserve the freedom to experiment without allowing every experiment to become an architectural commitment.
An experiment should have a question it can help answer. Is the interaction understandable? Does the visual approach communicate the intended information? Is a particular technique feasible enough to investigate further?
If I cannot name the question, the experiment can expand indefinitely. Every promising result becomes a reason to add one more feature.
An explicit question gives me a stopping point. Once I have the information, I can decide what belongs in the product and what should remain evidence of something we tried.
It also gives an agent a more useful assignment. A bounded investigation can report a result and its limits. An instruction to explore until something looks impressive has a much less reliable end.
My repository distinguishes experiments from production work and identifies shared sources of truth. That boundary is deliberate.
If an experiment becomes useful enough to promote, I want to integrate the idea with the established architecture: the owning parser or registry, the shared builders, the normal data path, and the relevant tests. I do not want the quickest experimental path to become a second permanent system by accident.
This is a place where agents benefit from clear rules. Without them, building another small editor or maintaining another list can look like the shortest route to completion. Locally, it may be. Across the product, it can create drift.
A prototype that reveals a bad assumption has done something valuable. So has one that shows the direction is promising but the implementation should be rebuilt inside the real system.
I try to separate those judgments. The experiment can succeed at producing knowledge while failing to produce code I want to keep.
That is useful for human teams as well. If every experiment must become a shipped feature to count as progress, people have an incentive to defend weak ideas instead of learning from them.
The record should preserve what we learned and what would need to change before trying again. It need not preserve every temporary mechanism as part of the product.
I like having the ability to try more approaches. I want to use that capacity to improve the decisions I make, not simply increase the number of half-integrated possibilities in the codebase.
Sometimes the right next step is a production task. Sometimes it is a better experiment. Sometimes it is to stop.
AI gives me more material to judge. I still have to decide what deserves to become durable.
]]>That surrounding work can absorb the benefit of a faster implementation. An agent may finish editing while I am still figuring out how to inspect the result.
I have put a substantial part of my AI effort into the tools around the product: editors, catalogs, previews, automated routes, capture utilities, and test harnesses. They give me more control over iteration. They also give agents a better way to participate than repeatedly searching the codebase and improvising a setup.
Consider tuning an effect. A normal playthrough might show it at the wrong scale, behind another object, or after a long sequence of unrelated actions. I can still judge it, but comparing two revisions becomes unreliable.
A focused preview lets me hold the important inputs steady and vary the thing I am trying to improve. A harness can trigger the same scenario, collect runtime output, and capture a result. Now “the second version is clearer” has a concrete reference.
The project history shows this tooling developing alongside the game. A web editor toolchain and server registry arrived in February. Automated encounter metrics and scoring followed in March. Later work added visual verification, clipping checks, capture manifests, and more focused regression tools.
These are different tools, but they serve the same purpose: shorten the distance between making a change and understanding it.
A preview that quietly uses different data from the game is worse than inconvenient. It makes the wrong result look authoritative.
I try to keep runtime behavior and tooling connected through shared definitions, manifests, and generated schemas. When a contract changes, the implementation, authoring surface, generated output, and relevant checks need to move together.
This is particularly useful with agents. Instead of asking one session to remember a list of valid names and another to maintain a second list, I can point both at the owning definition. The system does more of the remembering.
It does require judgment about abstraction. I do not want a universal editor with a hundred controls when the next task needs a scenario selector, one parameter, and a way to compare results. Good defaults and a few useful presets often buy more iteration speed than exposing every possible setting.
One September change captures the kind of improvement I mean. The warp-in effect catalog had tied its freshness to the overall HTML5 build stamp. That meant unrelated rebuilds could make the catalog appear stale.
The tool changed to track a narrower fingerprint of the inputs that mattered to its output. That made the freshness decision more precise.
There is a tradeoff here. A fingerprint that misses a real dependency can preserve stale evidence. Narrow invalidation needs a clear dependency model and a check that exercises it. But invalidating everything forever is not free either; it turns iteration into repeated work.
This is a useful question for AI workflows generally: what has changed since the last trustworthy result? If the answer is “nothing relevant,” I should not ask another agent to recreate the same proof.
It is easy to disappear into tools. A useful script can become a dashboard, then a platform, then a second product competing with the first one.
Before expanding a tool, I ask what repeated work it removes and who will use the result. If the setup is rare and cheap, a documented manual step may be enough. If the same ambiguity keeps returning across sessions, that is a stronger candidate for automation.
I also prefer extending an existing surface. Another one-off debug page gives me another place to maintain behavior and another chance for the tool to drift from the runtime.
The payoff I want is concrete: fewer steps to reach the state, less context to reconstruct, a clearer comparison, or an earlier failure when data is wrong. “We built an AI platform” is not a useful acceptance criterion.
The agent becomes more useful when it can operate a reliable experiment. I can ask it to change something, run the established path, inspect the output, and return with a specific discrepancy. I spend more time evaluating the result and less time directing setup.
Will Larson’s account of facilitating AI adoption is a useful organizational companion: tools and adoption need sustained work with practitioners, rather than an assumption that making a platform available is enough.
My experience is at the scale of a hands-on project, but the investment logic is similar. Improve the workflow people actually use. Keep the feedback close to the work. Make the next iteration cheaper without making the result harder to trust.
]]>The old decision depended on the overall HTML5 build stamp. Rebuild the game, and the catalog could need regeneration even when the inputs relevant to that catalog had not changed. The replacement used a narrower fingerprint of those inputs.
This is a small engineering detail, but it captures a bigger part of how I think about moving quickly with AI. A useful result has dependencies. I should know what invalidates it.
When a check passes, it establishes something about the code, data, configuration, and environment it exercised. It does not create a permanent property called “tested.”
If a relevant input changes, I need fresh evidence. If an unrelated input changes, the previous result may still be useful.
That distinction is especially important when agents can run checks cheaply enough to make indiscriminate repetition feel harmless. The cost can appear in build time, browser processes, resource use, or human attention spent reading another report that says the same thing.
The practical question is what new uncertainty a repeated check will resolve.
A narrow fingerprint is not automatically correct. If it omits an input that affects the output, the tool can report freshness while showing stale material. That is worse than being slow because it gives me a false basis for judgment.
So the optimization has a corresponding responsibility: make the dependency boundary explicit and verify that the freshness behavior follows it.
I want to understand both sides. What change should invalidate the result? What change should leave it valid? Those are useful scenarios for a tool check because they test the meaning of the cache rather than only its existence.
The same principle applies when an agent hands work to another agent. The next session should not have to rerun every check simply because the previous conversation ended.
It should know which state was tested, where the evidence lives, and what changed afterward. If the relevant inputs are unchanged, that evidence can carry forward. If a later edit affects the contract, the appropriate checks become necessary again.
This makes handoff quality part of validation cost. A report that says only “all tests pass” is difficult to reuse responsibly. A report that identifies the build and scenario is more useful.
I used to think about a test’s speed mostly in terms of how quickly it ran. Now I also think about how often it needs to run and how reliably I can tell when it is necessary.
That depends on architecture, source-of-truth definitions, and tooling. A tightly coupled system makes small changes invalidate large amounts of confidence. Clear boundaries make it easier to preserve what is already known.
The September catalog change did not prove that the entire project became faster by a particular factor. It removed one unnecessarily broad reason to repeat work.
Enough improvements like that can make the daily loop easier to sustain. I want agents spending their capacity on new information, with a precise reason when old information needs to be checked again.
This is one example of building tools that make the next iteration cheaper.
]]>That is an easy way to start. It becomes a poor way to decide how work should be organized.
In late August, I added more explicit director and implementation roles to the repository workflow. The purpose was to make delegation deliberate: who owns the change, what help is useful, and who decides whether the result is acceptable.
The question I ask now is what an additional agent would contribute that the current task cannot efficiently do on its own.
A read-only investigation can be useful while implementation is underway. One agent can inspect a neighboring contract, another can analyze a failure, and the writer can remain focused on the change.
That arrangement works when the findings inform different decisions. It works less well when three agents read the same files, produce similar summaries, and require me to reconcile the wording.
The same issue appears in review. A second look is valuable when it has a distinct responsibility: checking a runtime path, examining a risky assumption, or judging visual quality. A second agent that accepts the first agent’s explanation adds little independent evidence.
A contained fix often needs one implementation owner and a focused check. Making it pass through a planner, director, implementer, reviewer, and verifier can increase coordination without changing the quality of the decision.
My starting point depends on the work:
| Task | Useful initial shape |
|---|---|
| Known fix in one subsystem | One writer and the relevant check |
| Unfamiliar subsystem | A bounded investigation before implementation |
| Two independent changes | Separate owners with an explicit integration point |
| Consequential shared contract | A deliberate design decision and independent evidence |
These are defaults, not a requirement to create roles for their own sake.
I want more structure when the risk justifies it: shared contracts, consequential state changes, ambiguous design, or work that spans independently owned systems. I want the workflow to become simpler when inspection shows that the task is straightforward.
That requires the person directing the work to keep making judgments. A fixed number of agents for every task is easier to automate, but it is not automatically a good allocation of attention or compute.
The difficult part of a task is not always its largest diff. A small architectural choice can have more consequences than a long, mechanical implementation.
I prefer to spend more reasoning effort on unclear goals, competing designs, and decisions that are expensive to reverse. Once those decisions are made, a bounded implementation brief can let a less expensive worker proceed with useful autonomy.
The boundary is practical, not ideological. If the worker repeatedly needs help understanding the task, I may have chosen the wrong scope, supplied inadequate context, or assigned a problem that needs stronger reasoning.
Changing the model is one option. Improving the assignment is another.
Agents can run in parallel while their results still arrive at one human review queue. If I cannot evaluate the work promptly, increasing the number of writers can make the system less manageable.
I would rather leave capacity unused than create overlapping changes I am not ready to judge. I can also redirect capacity toward a bounded investigation that will make the next decision easier.
The useful number of agents changes with the task. I want each one to have a concrete contribution, a clear owner for the result, and a reason its work is worth integrating.
The guardrails essay covers the ownership rules behind these choices.
]]>