Human Spec - how to rein in Claude

TL;DR: Clearly state that some comments are “HUMAN spec” and tell your LLM to obey them, and keep them as is, and fix the code to conform to them if it doesn’t. Result: Much more controllable vibe coding.

Claude 4.0 (sonnet, thinking) seems to have a tendency to do a little bit too much.

For example, writing some FEM code, there were some functions that operate on sparse matrices. A test failed because someone was passing a dense matrix and instead of fixing the caller, Claude decided to be “helpful” and make the function also work on dense matrices.

All great and good except that in this application, having the caller generate the dense matrix would of course run out of memory immediately in a non-toy problem.

Now, this is a constantly reoccurring thing in LLM coding. The LLM gets some idea of its own It’s been trained to try to solve a problem (a friend used the term ‘implied “by any means necessary”’ and to not stop to ask for instructions too easily. Soon it’s doing something that’s not the right thing any more.

So, my current solution - I’ve been trying this for a little bit and have been very happy with the results.

In AGENT.md (or whatever file your system uses for the prompt), add a section

# HUMAN spec

IT IS IMPORTANT THAT YOU NEVER EDIT OR DELETE COMMENT BLOCKS MARKED AS "HUMAN spec".
THEY ARE TO BE KEPT AS IS.

Instead, check that the code that exists and the code that you produce
conform to the HUMAN spec as closely as possible. If the code does not,
fix it to conform.

In your final explanation of the changes you made, you are allowed
to suggest useful changes to the HUMAN spec. It is up to the user to decide
if they want to copy them to the spec.

Then, in your code, add HUMAN spec comments to make local rules about how you want that function / file to look like. Here’s an example from my codebase:

#
# HUMAN spec
#
# The tests are separate python functions. The tests shall not be unit tests
# but a single test that exercises a solver in multiple ways, parameterized
# by pytest.mark.parameterize to go through all solver types, with some kwargs
#
# There is a list of solvers to test (all solver types).
# This list is in the dict SOLVERS and can be used by external tests as well.
#
# The tests use a SparseMatrixLinearOperator, which is supported
# by all solver types (including lineax).
#
# The first test will test two aspects of all solvers:
#
#    1) Check that a 4x4 sparse matrix
#     [ 1 0 4 3 ]
#     [ 0 2 0 0 ]
#     [ 0 0 3 0 ]
#     [ 0 1 0 4 ]
#   is solved correctly for all 4 vectors in eye(4)
#
#   2) Check with a degenerate matrix that the solvers error out.
#
# Note: we're working with sparse matrices,
# so only test lineax iterative solvers
# such as GMRES
#

Prior to this, the generated tests were a mess of small, separate test functions, with some covering multiple solvers, some only one.

After this, the LLM has a clear goal and the goal doesn’t change because it decides to change it.

It’s really satisfying to see the LLM say

Looking at the `solvers_test.py`, the solver configurations are currently hardcoded
in the `@pytest.mark.parametrize` decorators. According to the HUMAN spec,
there should be a SOLVERS dictionary that can be reused by other tests.

Let me create the SOLVERS dictionary and update the tests to use it:

This solves the issue far more permanently than just prompting: just prompting may have the LLM undo its operations in the next session.

In some way, this feels like a “new” level of source code. You define the system by providing some small constraints from the edges and those “imply” most of the actual source code. You can even fold away the actual code most of the time and look at it only when necessary.

I haven’t been using this method for very long yet but I’m quite happy with it so far.