M. Semih Babacan

I Read DeepSeek's Agent Harness So You Don't Have To

DeepSeek open-sourced the program that wraps its model. The interesting decisions are all in that program, not in the model, and they are the same decisions you are about to make in your own AI product.

Mehmet Semih Babacan
Mehmet Semih Babacan

AI Technical Product Manager

August 22, 202610 min read22 views
I Read DeepSeek's Agent Harness So You Don't Have To

DeepSeek quietly published deepseek-harness under MIT, and the model is the least interesting thing about it. What they actually shipped is the program that wraps the model: the loop, the tools, the log, the rules about what the thing is allowed to touch. That program is where a product lives.

So I cloned it and read it properly, package by package, over a few evenings. If you ship anything with an LLM in it, the decisions sitting in this repo are the decisions you are about to make, whether you make them on purpose or by accident.

What a harness even is

A model is a function. Text in, text out. It cannot read your files, call your API, or remember what it said yesterday. Everything you think of as "the AI doing things" is done by the program around it.

That program is the harness. It holds four things the model does not have.

The model only ever sees text. Everything that makes an agent feel like an agent lives in the box on the left.

Swap the model and the product barely changes. Swap the harness and it is a different product. That asymmetry is the whole point of this post.

What is actually in there

There is no main() in this codebase. A running dsh is a tree of plugins assembled at boot from stacked YAML patch files, and the agent loop is a row in that tree. So is the model adapter, the tool registry, and the session log. You replace any of them with a one-row edit and no fork.

The cost is real and the repo does not hide it. Finding out who serves ctx.fs means grepping the whole workspace and then reading the patch files on top, which is why dsh --dump-config exists as a first-class command. When your architecture needs a tool to tell you what your architecture is, you have bought flexibility with legibility.

The session log is the truth, and the model's memory is derived from it. Every event is appended with a contiguous sequence number and nothing is ever deleted. What the model sees on the next call is folded out of that log by a function called deriveMessages(), not stored somewhere separate. The invariant they hold is: model-visible implies logged.

This sounds like plumbing. It is the most product-relevant decision in the repo. Any request can be reconstructed after the fact, tests replay recorded sessions without API keys, and when something goes wrong you have the actual sequence instead of a theory.

Compaction follows the same rule. When the conversation gets too long it appends a summary marked as replacing a range of events, and the old events stay on disk. The fold changes, the history does not, so your UI transcript survives while the model's window shrinks.

Every tool call goes through one guarded pipeline, and policy runs before execution, not after. Guards are deny-only by design: no guard can force-allow a call another guard denied. If a tool asks for approval and no approval service is mounted, the call is denied rather than allowed. That last detail is the kind of thing I look for. Fail-closed is a two-line decision that separates a system you would let touch production from one you would not.

Turns end for a reason and the reason is written down: completed, max-tokens, blocked, error, aborted. A closed set, appended to the log. Compare that to the usual "the agent stopped, probably fine."

Nothing reaches the model that was not written down first, and nothing runs that the guard did not see.

Steering lands mid-task, which I did not expect. New prompts queue as next-turn, corrections queue as next-step and reach the model at the next step boundary instead of after the whole task finishes. Anyone who has watched an agent confidently go the wrong way for four minutes knows why that lane exists.

Subagents are children with their own sessions, in-process or as a separate OS process. The genuinely funny part: DeepSeek ships providers that spawn Claude Code and Codex as subagents. They treat competing products as pluggable workers. Those providers register dormant and you have to copy a preset and remove a disabled flag to enable them, but the seam is there.

The repo says it was built primarily by coding agents, and the evidence is structural. Every decision carries a dated note with problem, decision, alternatives, consequences, each one mirrored in Chinese. CLAUDE.md is a symlink to AGENTS.md so one instruction file serves three different agent products. Rules are verify scripts in CI rather than paragraphs asking nicely.

Honest answer on maturity: it is 0.1.0-rc.8, a self-declared developer preview, on-disk formats pinned at version 0 with no migration path, no auth layer, and a BENCHMARK.md that amounts to "run it yourself." I read code and docs. I did not measure latency, token cost, or sandbox overhead, so I cannot tell you how it performs. Take everything above as an architecture review, not a benchmark.

What this means if you build AI products

The engine is becoming something you rent. Every frontier lab sells text-to-text by the token, prices keep falling, and any of them sits behind one adapter. That is evidence from this repo: the adapter is a row in a config file, and swapping it is a one-row YAML edit.

The inference I draw from it, and it is inference, is this.

The value is in the loop, not the engine.

the part I would defend in a product review

Your users experience your tools, your memory, your rules, your interface, and how well the loop is tuned. They do not experience the model directly. If your roadmap is mostly "wait for the next model," you do not have a roadmap.

Three consequences I would argue for.

Guardrails are a feature, not friction. Sandbox modes, one-shot approvals, explicit stop reasons, fail-closed defaults. The agent product that gets to touch production is the one whose guard the buyer trusts, and you cannot retrofit trust after an incident.

A durable log is auditability, and auditability is what companies buy. Consumer users rarely care that every decision is reconstructable. A compliance officer cares about nothing else. Building the log early costs a week. Adding it after you have sold to an enterprise costs a quarter.

Replaceability is survival, not elegance. If your product is welded to one vendor's API shape, a pricing change or a better model elsewhere is an existential event. If it is an adapter, it is a config change. This is advice for products whose core input is a fast-moving commodity. Do not generalise it to your whole business.

Where the analogy breaks: a harness has near-zero marginal cost per copy and its moat is trust plus integration depth, not capital or distribution. A restaurant swapping suppliers daily would die. A harness swapping models daily is doing its job.

The checklist I now run every agent design through

First question, and most teams skip it: do you need a loop at all?

A loop earns its cost when the next action genuinely depends on what the last one returned. If your flow is "classify, look up three things, write an answer," that is a pipeline, and a pipeline is cheaper, faster, and far easier to reason about. I have seen more damage from loops that should not exist than from pipelines that should have been loops.

If you do need one, the shape is the one in the second diagram above, and these are the questions I ask the team.

  1. What ends a turn, and is that list closed and written down? "It stops when it is done" is not an answer.
  2. What is the step cap, and what happens when you hit it? Escalating to a human is a fine answer. Silently truncating is not.
  3. Where does a human sit: auto, approve, or escalate? Draw the line by consequence, not by confidence score.
  4. Can a correction reach the model mid-task, or does the user wait for the wrong answer to finish?
  5. What is written to disk, and could you reconstruct any decision from it a month later?
  6. Which side effects are repeatable, and which would be a disaster twice?
  7. Can the model talk its way past your guards? If a guard is a prompt instruction, it can. If it is code, it cannot.
  8. When the context fills up, what gets dropped, and who decided?

Then there are five failure modes I check for specifically, because I have watched each one ship.

The first is a side effect that is not safe to repeat. Your queue retries on any non-2xx. If the send succeeds and the code after the send throws, the retry does it again. Anything that emails, charges, or posts needs a durable "I already did this" record, written before the call and checked before the next attempt.

The second is having no real record of what was decided. If your only audit trail is an analytics event that silently swallows HTTP errors, you do not have an audit trail, you have a hope.

The third is guards that live in the prompt. Instructions are suggestions. If the rule matters, it belongs in code that runs after the model, not in text the model reads and may argue with.

in the prompt
You must never run destructive
shell commands. Always ask the
user before deleting anything.
If unsure, stop and explain.
in the pipeline
registerGuard('tools/pre-execute', (call) => {
if (!isDestructive(call)) return 'allow';
return approvalService
  ? 'ask'
  : 'deny';   // no approver mounted, fail closed
});
Same rule, two places. The left one is negotiable, and the model is a very good negotiator.

The fourth is silent degradation. A helper that catches every exception and returns a default will keep your dashboard green while the product quietly gets dumber. Every swallowed exception earns a logged warning at minimum.

The fifth is evals that score the label and never the sentence. Most teams measure "did it pick the right category" and never measure "was the thing we actually sent correct." The second one is what the customer reads.

None of this is exotic. It is the same list DeepSeek arrived at, written down as architecture instead of as a postmortem.

What I took from it

Nobody using your product will ever meet your model. They meet the loop.

They meet the moment it stops and asks before doing something expensive. They meet the moment their correction lands mid-task instead of after four wasted minutes. They meet the moment you can tell them exactly what happened at 14:40 last Tuesday, because the log is still on disk and nothing was quietly overwritten. That is the product. The model underneath is the one part you did not build, and your competitor has the same one.

So the build order I would argue for is the one this repo already implies. Loop first, because a tight loop on an average model beats a loose loop on the best one, and users feel the difference in minutes. Guard second, because the buyer who lets you near production is buying the guard, not the reasoning. Log third, because trust is a claim until somebody outside your team can check it. The model seam last, so the week a better model ships you spend an afternoon on it instead of a quarter.

I would want those four settled and written down before anyone on the team writes the first prompt.