anti-patterns and patterns for achieving secure generation of code via AI
I just finished up a phone call with a "stealth startup" that was pitching an idea that agents could generate code securely via an MCP server. Needless to say, the phone call did not go well. What follows is a recap of the conversation where I just shot down the idea and wrapped up the call early because it's a bad idea.
If anyone pitches you on the idea that you can achieve secure code generation via an MCP tool or Cursor rules, run, don't walk.
Over the last nine months, I've written about the changes that are coming to our industry, where we're entering an arena where most of the code going forward is not going to be written by hand, but instead by agents.
Geoffrey Huntley
where I think the puck is going.
I haven't written code by hand for nine months. I've generated, read, and reviewed a lot of code, and I think perhaps within the next year, the large swaths of code in business will no longer be artisanal hand-crafted. Those days are fast coming to a close.
Thus, naturally, there is a question that's on everyone's mind:
How do I make the agent generate secure code?
Let's start with what you should not do and build up from first principles.
The first principle to understand when dealing with LLMs is determinism vs non-determinism. Security is one of those domains in our industry that requires a high level of determinism.
A lock is either locked or unlocked. Code is either secure or not secure. There is no shade of grey in this topic.
If you think that you can achieve security through offering guidance to the LLM through cursor rules, then you are misguided. Cursor rules or any of those types of rules (i.e AGENTS.md) that are attached to your agentic coding harness are mere suggestions to the LLM. They are suggestions.
It is non-deterministic. It is not security. It is an anti-pattern.
Geoffrey Huntley
Cursor rules are non-deterministic.
The next anti-pattern is any product or vendor selling a security solution that involves hooking the context window via a Model Context Protocol. MCP is a function with a prompt that provides suggestions to the LLM that the function should be invoked (tool called).
If you look at Model Context Protocol from the right angle, it is no different from Cursor rules. Cursor rules are also just a prompt. They're text in the context window, which are non-deterministically evaluated.
Geoffrey Huntley
a primer on MCP from an engineer who builds coding professional harnesses for a living
what you should do instead - outer loop
An outer loop is what happens during a pull request as part of your CI checks or before a git commit is pushed (ie, pre-commit checks or server-side push hooks).
There's a wealth of information available for many security vendors that provide SASTs, DST, and PBT-style tools. You should already have one by now, which automatically runs over any increment of change.
what you should do instead - inner loop
An inner loop is what happens during development. In the context of this blog post, this is where we're going to expand into guidance on how to drive an agent deterministically towards better outcomes.
Notice how I didn't say "secure code"? Whilst LLMs are good, I think that the idea that an LLM can generate secure code deterministically and for itself to decide what is safe and secure is just not possible and won't be possible for a long time.
What you need to achieve this outcome is simple. Take the command-line tool from your pre-existing security vendor and configure it as a deterministic hook.
There are a couple of ways of achieving this. Some coding harnesses, such as Claude Code, support inference hooks which allow you to hook the inferencing loop.
But what happens if not everyone in the company is comfortable using Claude Code, and some use Cursor? This is the conundrum. Every company out there is currently evaluating various coding assistants or building their own.
Geoffrey Huntley
this workshop teaches you the inferencing loop, tool registration and how to build your own agent from first principles
The best way to do it, because security needs to be deterministic and absolute, is to hook any coding agent via your compilation target.
in practice
I've written about this before, but there are two phases in AICG - generate and backpressure:
- The generation phase is where you put your suggestions to the LLM of what you would like to be generated and how it should be generated. This phase is non-deterministic.
- The backpressure phase is where you validate against hallucinations, and what has been generated is successful. This phase is deterministic.
Below you'll find a Makefile. It doesn't have to be a Makefile. It could be a target in your package.json, or it could be a bash script; it could be anything, really. It just needs to be your build target.
.PHONY: all build test
all: build test
build:
@echo "Build completed successfully at: $$(date)"
@exit 0
test:
@echo "Tests completed successfully at: $$(date)"
@exit 0Inside your AGENTS.md you should put instructions that the agent should invoke your build target after every change. This is what creates the backpressure to the generation phase.
# Agent Instructions
## Code Quality
After every code change, you MUST:
1. Run `make all` to verify that the code builds successfully and tests pass.Let's open up a coding harness, in this case, Amp, and ask it to generate a FizzBuzz application and run the build. The agent will read the AGENTS.md file and learn how to run the build, which will be executed automatically after the Fizzbuzz application has been generated.
simple right?
Okay, so let's dial it up a notch. If you take any of your existing security scanning software and hook it in as a target, then guess what happens? The security scanning software will run automatically every time code generation is complete.
So let's update our Makefile with a new target called security-scan and update the default target (all) to run it.
.PHONY: all build test security-scan
all: build test security-scan
build:
@echo "Build completed successfully at: $$(date)"
@exit 0
test:
@echo "Tests completed successfully at: $$(date)"
@exit 0
security-scan:
echo "Security scan completed at: $$(date)"
@echo "Code is insecure!"
@exit 1The next step is to update our AGENTS.md to be a little bit more prescriptive so that it resolves security issues identified by our deterministic security scanning tool.
# Agent Instructions
## Code Quality
After every code change, you MUST:
1. Run `make all` to verify that the code builds successfully and tests pass.
2. IMPORTANT: You MUST resolve any security issues identified during compilation. Let's drive in a loop and see what happens with these updated instructions...
Notice how the security scan target is now invoked, and the agent (regardless of which agent you use, whether that be Amp, Claude Code, Cursor, RooCode, Cline, or anything) will now take the suggestion and ponder about generating a new variant of the code that was previously generated.
why this works
It's not the agent that does this behaviour; it's the underlying LLM model.
If it's in the context window, then it's up for consideration as a suggestion that it should be resolved.
As compilation is a mandatory step in the SDLC, it is deterministic that the security scanning tool will be invoked after the code generation phase.
If the Security Scanner tool exits with a non-successful return code, then the output of that tool will be evaluated by the LLM in the next inference loop. When it sees the problem, it will do its best effort to resolve the problem identified by the Security Scanning tool
so now what?
The answer is simple. Take your existing security scanning software and configure it into your build target, send a pull request in, get it merged. Then, any autonomous agents will automatically be nudged to try again when your security scanner detects that a potential security violation has been tripped.
big brain mode
This is a generalised pattern. Folks, you can use this to ensure that bad patterns within your codebase, regardless of whether they're security-related or not, are not proliferated by coding agents. All you need to do is put your engineering hat on and configure some language analysers.
closing thoughts
Secure code generation is a misnomer. Security is a practice and technique, not a product.
These LLMs, although trained on security topics, are unable to make decisions on their own about whether something is secure without some external guidance from a deterministic system.
Even if code generation via MCP as a form of a security tool was a plausible approach to solving the problem domain, it makes little sense to purchase yet another security tool and then have security teams maintain yet another rules database when there is already a rules database in the outer loop.
If you want good outcomes with AI code generation that is "secure" (aka better than nothing), then just adapt and use the same rules database used in your outer loop in your inner loop, then focus on making your security suite fast.
p.s. socials
