Articles / Engineering · September 20, 2026

A report I can plan from

Before I can use an agent’s report to plan work, I need to see which claims have evidence and which are still guesses. That’s what I wanted from plan-explore: a way to take a brainstorm, investigate the codebase, and turn the findings into a report I could check.

1. The questions I kept repeating

I started by writing plans with an LLM in the web UI, copying them into a notepad, and working through each phase in a fresh context. When I started using Claude Code, I began keeping those plans locally in Markdown.

I was still guiding the agent through the same questions, though. After repeating that guidance enough times, I put it into a skill. That became plan-explore: take an idea, investigate what already exists, and produce a factual report I could use as the base for a plan.

I needed room for that investigation before deciding what work to ask for. A brainstorm might tell me what I wanted to explore, but I still had to understand the system I was asking an agent to change.

2. Saving the questions didn’t fix the answers

When I went back through the reports, I kept finding things that were false. I remember an agent assuming we used RSpec where we used Minitest. It would also make assumptions about how parts of the system worked.

An answer about our tests could sound reasonable while starting from the wrong conventions. I had to check what the agent was basing its answer on before I could use it. Putting my questions into a skill made the guidance reusable, but I was still going back through the answers and challenging the claims myself.

That back-and-forth became another piece of work I kept repeating. I could give another agent the job of questioning the report.

3. Giving another agent the review

I started splitting research into focused questions so each subagent could work with a tighter context. I remember using a model like Haiku for that research. Then I could ask other agents to review what they found. When I began asking an agent in a fresh session to identify inferences and shaky claims, those labels became part of the report.

The instructions checked for this article specify Sonnet for research. That is the configuration described here; Haiku is part of my recollection of how I got started.

The work is divided this way in those instructions:

• Researchers gather the material. The code researcher reads the relevant files, checks routes and names against the codebase, and reports mismatches. Other research covers documentation and, where it applies, user scenarios. The findings go into a briefing for the next stage.
• An Analyst puts the report together. It cross-references the research and flags claims it cannot verify.
• A Critic questions the report. It challenges high-confidence claims, asks for evidence, and looks for untested assumptions and missing scope. It can propose corrections to the Analyst, and its validation summary is supposed to accompany the report.

I was trying to give agents the repeated review work I had been doing myself. The instructions ask them to flag unsupported claims, but they can still miss a mistake or leave an inference unlabeled.

The workflow also allows one agent to assemble the report when teammates are unavailable, without a separate agent challenging its findings. A saved report by itself doesn’t tell me whether the full review happened.

4. Following a guess through the report

Let’s look at a September 4, 2026 discovery report about posting to Bluesky. One question was what would happen if a posting request were retried with the same record key, the identifier assigned to that record.

The report gave one answer, then corrected it later. These are condensed paraphrases of those answers; the inference marker is quoted exactly:

We can follow the change in reasoning here. The first answer was a guess about an overwrite. The later answer says the add operation rejects the key, and it leaves the client response open. That remaining question matters if I’m going to build a posting tool around the result: what response does the tool actually need to handle?

A captured excerpt of the official tree implementation, checked on September 20, 2026, supports the correction about the add operation. It throws when it finds a leaf with the same key. That tells us what this operation does; the excerpt alone doesn’t tell us what response a client gets from a deployed server.

Q16 also recommends treating an already-exists failure on retry as proof that the post landed. The source check doesn’t establish that. Before using the recommendation, I would need to confirm the client response and whether that failure is enough to count the retry as a success. Otherwise, I would be building the next step around another assumption.

The correction is useful, and so is being able to see where the evidence stops. The report doesn’t identify who made that correction, so this example can’t tell us whether the Critic caught it.

5. Starting with the report

The next thing I want to build is a new exploration skill. I want to start by deciding what a good exploration report needs to contain, then work out an efficient way to produce it.

The current report format gives me a starting point. Its schema requires evidence and confidence alongside each answer, with places for unknowns, assumptions, risks, and conflicts. The instructions call for structured JSON and a readable Markdown report generated from it. Those fields give me places to look, but I still have to judge whether the answers are supported and useful for planning. Recommendations in the report are proposed work; they don’t establish that anything has been implemented.

For the new skill, I want to define quality in terms of what the report helps me do. Can I understand the relevant parts of the system? Can I follow the evidence behind a claim? Can I see which assumptions could change the plan, and whether an unanswered question needs more investigation before work begins? Those are the things I want to evaluate when comparing reports.

Then I can compare ways to get there. A single agent, focused researchers, and a separate review step are options to test. I want to try them on the same exploration tasks and look at the quality of the resulting reports alongside their cost, time, and the amount of checking or correction they still need from me. Another agent or review pass should earn its place through a useful improvement to the report.

The material examined for this article doesn’t measure whether the current workflow reduces factual errors or improves ticket outcomes. For the next version, I want to find a balance of reasonable cost and reasonable expected quality, using those comparisons to guide the design.

When I was asked what made a report reliable enough to plan from, I put it this way:

Transparency is what made the report reliable enough. Knowing what’s not a fact.

That is one requirement I want the new skill to keep.