I build tools so the next iteration costs less
When I want to improve an interaction in my game, writing the change is only part of the work. I also have to get back to the right state, trigger it, see what happened, and compare it with what I intended.
That surrounding work can absorb the benefit of a faster implementation. An agent may finish editing while I am still figuring out how to inspect the result.
I have put a substantial part of my AI effort into the tools around the product: editors, catalogs, previews, automated routes, capture utilities, and test harnesses. They give me more control over iteration. They also give agents a better way to participate than repeatedly searching the codebase and improvising a setup.
Build a way to ask the same question twice
Consider tuning an effect. A normal playthrough might show it at the wrong scale, behind another object, or after a long sequence of unrelated actions. I can still judge it, but comparing two revisions becomes unreliable.
A focused preview lets me hold the important inputs steady and vary the thing I am trying to improve. A harness can trigger the same scenario, collect runtime output, and capture a result. Now “the second version is clearer” has a concrete reference.
The project history shows this tooling developing alongside the game. A web editor toolchain and server registry arrived in February. Automated encounter metrics and scoring followed in March. Later work added visual verification, clipping checks, capture manifests, and more focused regression tools.
These are different tools, but they serve the same purpose: shorten the distance between making a change and understanding it.
Give the tool a source of truth
A preview that quietly uses different data from the game is worse than inconvenient. It makes the wrong result look authoritative.
I try to keep runtime behavior and tooling connected through shared definitions, manifests, and generated schemas. When a contract changes, the implementation, authoring surface, generated output, and relevant checks need to move together.
This is particularly useful with agents. Instead of asking one session to remember a list of valid names and another to maintain a second list, I can point both at the owning definition. The system does more of the remembering.
It does require judgment about abstraction. I do not want a universal editor with a hundred controls when the next task needs a scenario selector, one parameter, and a way to compare results. Good defaults and a few useful presets often buy more iteration speed than exposing every possible setting.
Invalidate the evidence that actually changed
One September change captures the kind of improvement I mean. The warp-in effect catalog had tied its freshness to the overall HTML5 build stamp. That meant unrelated rebuilds could make the catalog appear stale.
The tool changed to track a narrower fingerprint of the inputs that mattered to its output. That made the freshness decision more precise.
There is a tradeoff here. A fingerprint that misses a real dependency can preserve stale evidence. Narrow invalidation needs a clear dependency model and a check that exercises it. But invalidating everything forever is not free either; it turns iteration into repeated work.
This is a useful question for AI workflows generally: what has changed since the last trustworthy result? If the answer is “nothing relevant,” I should not ask another agent to recreate the same proof.
Tooling has to earn its maintenance cost
It is easy to disappear into tools. A useful script can become a dashboard, then a platform, then a second product competing with the first one.
Before expanding a tool, I ask what repeated work it removes and who will use the result. If the setup is rare and cheap, a documented manual step may be enough. If the same ambiguity keeps returning across sessions, that is a stronger candidate for automation.
I also prefer extending an existing surface. Another one-off debug page gives me another place to maintain behavior and another chance for the tool to drift from the runtime.
The payoff I want is concrete: fewer steps to reach the state, less context to reconstruct, a clearer comparison, or an earlier failure when data is wrong. “We built an AI platform” is not a useful acceptance criterion.
This is how I get more out of the same models
The agent becomes more useful when it can operate a reliable experiment. I can ask it to change something, run the established path, inspect the output, and return with a specific discrepancy. I spend more time evaluating the result and less time directing setup.
Will Larson’s account of facilitating AI adoption is a useful organizational companion: tools and adoption need sustained work with practitioners, rather than an assumption that making a platform available is enough.
My experience is at the scale of a hands-on project, but the investment logic is similar. Improve the workflow people actually use. Keep the feedback close to the work. Make the next iteration cheaper without making the result harder to trust.