Humans must be tightly integrated into the practice of Harness engineering. Non-deterministic LLMs drift with high level thinking.
What?
High level and high order thinking do not seem to be within the capabilities of the modern LLM models (as of Opus 4.8).
As of writing this, Opus is well regarded as an extremely intelligent and capable LLM model that can solve very complex tasks. We use it to write software and everyone is going through these moments of realization that AI is insane (or whatever term it is that they use to show their disbelief / acknowledgement). It’s true, Opus can tackle most programming tasks and write code. It’s incredible. It wowed me in May of 2025. It wows me today.
Opus is not a model that can execute high order thinking. It can’t build a harness for a software application with a simple one line prompt. If you tell it to build a harness, you’ll get something, but you probably won’t get something valuable. It requires additional context. We can achieve higher level hands-off approaches through refinement of new context and automation processes.
AI is exceptional at writing something out that has a lot of tokens / words. It is not exceptional at thinking at the critical level that is needed to push through the frontier of the AI revolution we are in. We should continue to operate on a basis that humans will provide the steering of the context & harness while agents provide the majority of the steering of the application. This should remain until frontier models become intelligent enough for us to step back and become more high level.
Everything must anchor to human intelligence
My philosophy, the concepts I discuss, and the ideas I come up with are all drawn from my cognition; We are the exemplars. I think and I write. LLMs have not been able to successfully write about harness engineering. They aren’t effective at assisting me in writing this content. They are, however, adept at helping me work through ideas. They ideate with me and help me understand the directions I could go.
They struggle at writing because they always want to embellish it with things they are aware of that they think are related. They always want to add thoughts. They drift a little bit in every response. They want to AI’ify it.
AI’ification when comprehension is not there seems to drift at a higher rate. It washes it out with the Average content of the internet.
This is why all of this content is as close to human output as I can tolerate (I just can’t pass on having AI fix grammar and spelling, give me a break).
How AI helps me with the theory of harness engineering
I use an LLM as I work in this repository of information. It has been an incredible tool. But it doesn’t write the content itself. I’ve tried to have it write at this high level of thought but I cannot find a way to have it write the concise concepts I’m trying to get across. Here is how it does help:
- It does basic grammar, spelling, and writing review
- It manages cross links and identifies overlap of concepts
- It reviews the tone and structure to maximize Readability and skimmability
- It helps evaluate whether the content is too theoretical or without base
- It generates diagrams and keeps their styles consistent
- It does basic linting, checking for consistent casing in titles, matching titles with filenames and indices
- It checks tone - opinionated, but measured
- It harshly criticizes in a way humans may not
The bias against AI-generated content
Humans seem to be adept at spotting content that they suspect is AI written. The general vibe is that content generated by AI is often discarded or ignored. The bias seems to have grown off the rise of AI slop. This is partly why I hold my philosophy: We’ve all learned very quickly that AI can go off the rails. We have recognized patterns that we can identify as “This is obviously not coming from high order thinking.” We can identify when drift has compounded.
Sometimes I do write like AI. I write markdown files. This is because I’ve learned that we are being exceptionally well trained at taking in, skimming, and reading markdown content that AI delivers. LLM output is influencing how I write. Opus 4.8 in its verbose and eccentric vocabulary will grow on me.