Harness is an appropriate name. It comes from reinforcement learning world where you need to build the proper scaffolding for it to optimize for the goal you want it to.
This is very similar to what the agent is doing. You are building the appropriate environment for it to be able to complete the task most reliably etc
Not just functions/tools and documentation available (which is similar to a library), also context and critically, enforcement of behavior.
This is probably the key thing that makes it a "harness". If the agent can do whatever it wants, it's not in a harness.
You can ask AI to focus on the functional aspects and create a design-only document. It can do that in chunks. You don't need to know about COBOL best practices now, that's an implementation detail. Is the plan to modernize the COBOL codebase or to rewrite in a different language?
First off, what you shared is cool, thank you. Especially considering it captures problems I need to address (token limitations, context transfer, managing how agents interact & execute their respective tasks).
My challenge specifically is that there is no real plan. It feels like this constant push to use these tools without any real clarity or objective. I know a lot of the job is about solving business problems, but no one asking me to do this has any idea or defined acceptance criteria to say the outputs are correct.
I also understand this is an enterprise / company issue, not that the problem is impossible or the idea itself is bad. Its just a common theme I am seeing where this stuff fails in enterprises because few are actually thinking how to apply it... as evidenced by the fact that I got more from your comment than I otherwise get attempting to collaborate in my own organization
Spec-driven development is the only reliable way to work with AI. That's my current understanding. I spend more time refining the spec and bouncing ideas off of AI/team than before, which is good before there can't be any incorrect assumptions or hidden variables, otherwise AI will create suboptimal code. We should have been doing this much earlier in the process, even without AI, but now it's more necessary than ever. If you keep asking AI to make small changes as you learn about the business domain of your project, it will create a mess, in my experience. It's better to start from scratch and ask it to reimplement, if you finally understand all the requirements.
Sentiments like this make me wonder if perhaps the dream of the 90s was just ahead of its time. Things like UML, 4GLs, Rational were all being hyped. We were told that the future was a world where people could express the requirements & shape of the system, and the machines would do the rest.
Clearly that didn't happen, and then agile took over from the more waterfall/specs based approaches, and the rest was history.
But now we're entering a world where the state of the art is expressing your requirements & shape of the system. Perhaps this is just part of a broader pendulum swing, or perhaps the 1990s hopes & dreams finally caught up with technology.
Yes and no I'd say.
It's still the case that now only by iterating and testing things with the AI you get closer to an actually good solution.
So up front big spec will also not work so well.
The only exception maybe if you already have a very clear understanding and existing tests (like what they did with the Claude's building the rust c compiler to compile the Linux kernel)
I think PG said something about sitting down and hacking being how you understand the problem, and it’s right. You can write UML after you’ve got your head round it, but the feedback loop when hacking is essential.
This likely won't need billions of Euros to implement and will be an earmark in the budget. My point being it's not such a grandious project, from a continental perspective.
I think you should have added a disclaimer that you are the founder of company that provides "Reliability and context for complex environments."
It feels a bit dishonest to be asking for advice on how to tackle the complexity problem for SREs when you're are actually providing a paid solution for the very same problem.
I'm seeing this pattern pop up more and more all over the place now. It's pervasive throughout Reddit too for example: pick a sub in the area that you built your app in, pose some problem, and then have another account also controlled by you present the solution that you built. All the writing styles in these posts are similar too; it's all likely written by AI, including the post we're commenting on.
reply