There are a lot of AI coding agents out there, but they all stall out if you don’t tool your codebase for them. There’s a ceiling of how effective they can be if they’re running in a boilerplate environment, with no customized context or guidance.

I now run a consultancy helping companies get more from these tools. To show what tooling an AI codebase looks like, I forked Apache Superset, a popular BI tool, and implemented each one of my recommended changes there for you to use an an example.

Give it a sandbox to run in

Running AI on your computer comes with worries. Can it access credentials to drop prod? Might it accidentally delete your whole hard drive? If you’re running the agent directly on your machine, you have to baby sit every iteration it does, because the consequences of a bad action are so high.

On the flip side, agents are the most useful when they can iterate “forever”. Searching the web, trying out code, fixing errors, running tests, evaluating output. If you have to approve every step, this is a non starter.

My tool of choice here are devcontainers. They’re a way to define a docker container with some extra sugar to allow most of the major IDEs to run against the container. Being in this sandbox gives you the safety to let the agents run wild, doing the most work. The fact it integrates so broadly with IDEs gives a huge leg up over VMs or other tools in the area.

Here’s it in action with Superset

Feed it context on session start

It can be frustrating working with an agent, only to start a new session and have it completely forget everything it learned about the project (IE read into context). Creating documentation for the AI is key, and tooling the agent to read that context in at session start is imporant.

Most projects I work with now have a root level docs folder, usually a sphinx project that are detailed project instructions about the development nuances of the codebase. This covers project structure, testing stratgies, and other useful context. I then instruct in CLAUDE.md or .cursorrules to have the agent review this context on start.

You can even use the agent to maintain the docs. At the end of every session instruct it to update relevant data, making sure the docs act as a “long term memory.” The code will always be the source of truth, the docs are a more abbreviate project stucture to help boostrap your sessions.

Here’s the agent instruction

Here’s the documentation

Be clear on how it should verify its work

The feedback loop is the most important part of agentic development. Every change the agent makes should be verified by tests, linters, and compilers. The faster the agent can get feedback it was wrong, the less likely it is to get off track.

In the same rules files as your context injection, outline expectations of what commands should be run when. For instance if you have a test suite that can run a specific file, make it clear the agent should just test the file on change, and then test everything before committing.

I’ve found creating a root Makefile with agent specific tooling calls is very helpful, as the agent can read in that one Makefile and immediately have context on every action it can take.

Here’s an example where I give detailed instructions how to operate the UI

A prompt library for plan, act, commit

Agents are like humans in that they’re way more successful if they have a plan to follow. These plans have best practices, and encoding those best practices in prompt templates are very useful, EG commands for Claude Code, notepads for cursor.

Every project I work on has 4 key prompts

• Make a plan: A prompt with “few-shot” templates for what good plans look like. Include step by step tasks, demand it outline unkowns, and have sections for security and performance concerns.
• Execute the plan: A prompt to act on the “next step” in a given plan. Thoroughly outline general expectations, such as always adding tests for new functionality.
• Update the docs: Capture changes in our long term memory documentation.
• Commit the code: Bundle outstanding changes and create a logical commit message based on the changes and the session history.

Command examples

Extend the functionality

Agents are LLMs with capababilities. Coding agents will provide tools like reading and writing files, executing shell commands, and maybe fetching from the web.

MCP allows these agents to do much more. They can search databases, get production logs, order a pizza. What makes sense to use is very codebase dependant, but there are two I’ve started to use in every project:

MCP Pupeteer: Allows your agent to open a web page, click around and take screenshots. This unlocks a new level of feedback loop when building web UI, as it can check the final rendered page for the presence of elements, and even behavior from clicking.

Context7: One of the most annoying things about using agents for coding is fighting with the knowledge cutoff. Maybe the library you’re using is newer than the models’ knowledge, or there are just not enough training examples. This MCP allows you to easily ingest up to date documentation for all your dependancies for any version, without having to “crawl” the documentation webpages.

In the superset context

Closing thoughts

AI Agents have foundationally changed how I thinking about programming computers. What used to take months can take days now. You need to understand and optimize these agents for your usecase. They’ll provide some value out of the box, but they really shine when you take the time to understand how to get the most out of them.