A classifier that
returns numbers.

Jev is a System One model from TypeSafe. It does not write text. You send it a state object and a set of questions. It returns a typed answer for each one, all read off that one state. Three answer shapes exist:

• Choice: pick one of up to 255 options. Returns the choice, a probability for every option, and a confidence.
• Score: rate against two to ten ordered levels. Returns a possibly fractional score, per-level probabilities, and a confidence.
• Noul: yes or no, as a bare probability between 0 and 1. It has no confidence field. A gate must not wait for one, because the model never sends it.

Output is free and input is $0.042 per million tokens. A decision costs whatever it takes to describe the state. The answer itself adds nothing. Questions are batched for this reason: several questions on one state cost the same as one.

The advisor is first-party SpecPi code that answers questions about session state and never holds authority: it cannot call a tool, grant a capability or allow an action. The command guard beside it is a separate pinned package, specpi-lancet-guard, and the one exception: it scores commands before they run, and can refuse one. It does not use Jev or a key, and it scores commands without the network.

A call reaches the network only after passing every gate below, in order. If any gate says no, the attempt ends quietly and the harness follows its normal path. No advice is always an acceptable outcome. A timeout, a missing key, a refused consent, or a low confidence answer all look the same from outside.

Consent is separate from the switch. A config flag records a setting. It cannot record that a person was told what it costs. The first time any system would send state, a dialog names the destination host, the shape of the data, and the byte limit. Without a person at the keyboard, nothing is sent. An unattended run cannot start transmitting. The grant is tied to the host it was given for. Pointing the client elsewhere asks again.

One rule covers all five: check local state first. Spend a Jev call only on a question local state cannot answer. That rule removed the largest measured waste here. Six of SpecPi's ten tools were offered on every request and never called. Two of them need a human to select an improvement first, which is already recorded on disk. They stay hidden until one exists.

• Retention decides whether a large listing, search or fetched result is still worth keeping, before it is saved. File reads are not asked about: the agent wanted them. Jev judges relevance and whether it contains the answer. Code makes a fixed head-and-tail digest only when it saves bytes. No model-written text enters the transcript.
  See the flow
• Gap adds advisory cluster, impact and fix suggestions to locally collected reports. Original impact, priority and canonical keys stay unchanged. Humans decide which reports to merge or select. Its one refusal: a report that writes out a secret, or names a specific person or machine, is sent back to be rewritten.
  See the flow
• Sources sorts the sources a delegation batch will freeze. It reorders only. It never drops one. It also warns if a read-only child looks unable to answer the job's question; every selected source stays selected.
  See the flow
• Untrusted content adds a fixed warning line to a fetched page that reads like instructions for an agent. It covers web and browser results and pages fetched through the shell with curl, wget and similar, never other shell output or file reads, and it never blocks. When the same result also qualifies for retention, both questions share one call.
  See the flow
• Capability arming checks the request at the first enabled, interactive opportunity, at most once per session. It offers a hidden tool group only with human approval. Enable it before starting to get an early proposal; enabling it mid-session can prompt later. Section 06 shows why that timing matters.
  See the flow

One way is to let results pile up, then rewrite them in a batch. Simulation over recorded token history estimates 12% lower long-attempt cost for a batch rewrite, versus 61% for shortening on arrival. These are modelled savings, not observed advisor benefits. A rewrite throws away the cached prefix, and about 94% of prompt tokens are cache reads.

Every choice above follows one claim: changing the tool set mid-session drops the provider's cached prompt prefix. About 94% of prompt tokens here are cache reads priced at a fiftieth of fresh input, so that drop moves most of the spend. That is why there is no tool router, why capability arming is intended to run early, and why advice is never applied to past turns. Enabling capability arming later can still prompt mid-session.

This had not been measured here. It matched Pi's docs and general reasoning, but no number backed it. The test used three arms of t3-cascade-ledger with one difference: when Browser QA's fourteen tools reach the request. Never, at turn six, or from the first request. The rule was written down before the run: if cached tokens drop and re-warming costs more than 10% of attempt cost, the rule stands. Otherwise it goes.

The result cleared the bar. In all three attempts, the request after the switch saw cached tokens fall to 3,200 (from 25,984, 17,792 and 21,120) while the prompt kept climbing. One request went from 3,673 fresh tokens to 31,337: the whole conversation so far, re-read at fresh-input prices. The re-warm cost 14.6%, 21.6% and 23.9% of the attempt, 20% on average against the 10% bar set in advance.

Arming the same group from the first request cost 16% more than never arming it. Switching it on mid-session cost 47%. Asking up front cost much less than switching later. That timing gap is what capability arming exists to use.

Limits. Three attempts per arm show an effect this size but cannot rank the arms closely. The turn-1 arm also took more turns than the control (18.7 vs 16.3), and some of its extra cost comes from that, not schema size. The plan for this experiment also said activating Browser QA rebuilds the system prompt as well as the tool schema. That is no longer true: the pinned 0.3.0 release moved its guidance into tool descriptions, and the measurement confirms it. System prompt length matches between the control and armed arms. The old claim is removed. Web access, a third-party package, still carries prompt metadata.

The gate turns a probability into a decision. Until now every number in it was a placeholder. The layer shipped before its own gate was set. Calibration asked two questions, because a threshold can fail in two ways.

Does confidence tell right from wrong? Tested on labels this repo already had: predicting an attempt's outcome, task type, and tier from behaviour alone, with the labels hidden. For a Score the answer is yes, weakly: about two thirds right and 96% within one level, on a fifth of answers, against a 43% majority. For Noul and Choice on that set, the answer is no: both stayed near their base rate. No pre-registered precision target was met on those curves. The artifact records that instead of a fitted number.

Can the gate fire at all? This question found a real defect. The retention gate needed Score confidence of 0.80 and a value within 0.15 of a level. On a clear-cut case (a directory listing of vendor icons during a changelog edit), Jev answered “spent” every time, at confidence 0.73 to 0.85, mostly under 0.80, with the value drifting past the line too. So retention could approve keep this result but almost never this result is spent. The branch that does anything was out of reach. Running the layer longer would not have shown it, because a silent system and a system with nothing to say look the same.

The same check caught a second gap in compaction, and a third in new code for this release: the capability threshold sat above what the model sends even on a clear yes. Each fix read the working point off recorded results instead of picking a cautious-looking number. Thresholds are now pinned by a test to evals/runs/jev-calibration.json, with the figures quoted on this page. Changing one needs new evidence.

On the live questions, where the state holds the material under review instead of counters, the gap is wide: a planted credential scores 0.96 vs 0.04 for a clean report, a page with an injected instruction 0.97 vs 0.04, a thrashing session 0.89 vs 0.30 for steady work. The gates sit in that gap.

The command guard is not one of the five systems above, and it does not use Jev. It is specpi-lancet-guard, a separate pinned package with its own gate, its own configuration file and its own commands, and it is the only thing SpecPi installs that can take a tool call away. It scores commands with LANCET, a small classifier that runs on your CPU: no key, and nothing leaves the machine. It ships off, and SpecPi never writes its configuration. /lancet-guard setup downloads and verifies the model and offers to turn it on; /lancet-guard on does it for a session and --global saves it. It replaced specpi-jev-guard, which scored commands with Jev through OpenRouter. The LANCET guard page covers it in full.

It fails closed, and that is the trade to understand before switching it on. With the model missing or damaged, the commands the rules leave open do not go through. In a session with no one to ask, an ask blocks too unless uncertain is set to allow. @gotgenes/pi-permission-system decides every call after it, and every call while it is off.

• Local rules first, free. Read-only commands and ordinary project writes never leave the machine. A hard-deny list covers the clearly catastrophic cases — deleting the filesystem root, writing a raw disk device, piping a download into a shell — and your own safeCommands, allowedCommands and disallowedCommands are consulted before anything is sent.
• LANCET scores what is left. A command it flags as risky asks you (or blocks, in /lancet-guard mode block); one it does not flag runs. It reads Bash, PowerShell and cmd; commands over 8,192 bytes always ask, as do commands longer than one 512-token window that it does not flag. /lancet-guard check scores a command without running it.
• Writes are judged by target. Ordinary project files pass. Paths outside the workspace, and paths matching protectedPaths (.env*, keys, .ssh/), ask.
• Once a person is asked, silence means no. A dismissed dialog is not consent. A question with no answer cannot resolve to yes.

The guard publishes a per-session counter — commands LANCET scored, blocked calls once there are any, and the last verdict. In the terminal it is a footer line; in SpecPi Chat it sits in the session footer beside the token and cache readouts, showing the count and turning amber once something has been blocked, with the full line in the tooltip. Chat renders the package's own string and nothing more: it is a readout, not a control, and it disappears when the guard is off. Every judged call is also written to Pi's session file, which is the durable audit trail. /lancet-guard audit off turns the line off without touching that record.

Updating from a SpecPi that pinned specpi-jev-guard retires that package's entry. If it was switched on, the update says so: the LANCET guard does not inherit that setting and starts off.

The layer's row has been re-run since the gate was set, so there are two Jev columns. The middle column is the first full matrix (every harness, every tier, one model, one sitting) where three systems were on and retention could not shorten anything. See section 07. The right column is the same 37 attempts re-run with the set gate and five systems on, the setup described on this page.

The layer has not paid for itself. It costs about 9% more per attempt and scores close to the base either way. The one figure that once suggested the mechanism was working (context up 6% less per turn) did not hold on re-run: 590 became 631 against a control of 628. That was noise, and the first reading was too kind. The tool error rate is lower in both Jev columns, but nothing here touches a tool call before it runs, so there is no mechanism behind that link. It is not claimed as one.

The ledger now records each call: was the advice used, and what did it save. The eval runner reads that out of each attempt's folder. Over the same 37 attempts the layer made 43 calls, none failed, none changed anything. They cost $0.0013 in total, 0.77% of the run's model spend. Call price is not the problem.

Retention asked 38 times and acted zero times. That is now a finding. The first time, the gate was out of reach, so silence proved nothing. This run used the set threshold, where a test spent result (a listing of vendor icons during a changelog edit) answers 0.75 to 0.85 and would pass. The gate opens. On 38 real reads the model stayed below it each time. On the real reads of a repair chain, where each result feeds the next step, it reports doubt, and the record shows that.

The threshold stays where it is. Keeping a result costs tokens. Dropping the wrong one can cost the task. Acting without confidence is what this system must avoid. The number now supports a plain claim: on this workload the main system does not pay off. Sessions with truly disposable output (wide greps before settling, pages read once) are still untested.

The ledger showed two more things a cost column hides. Progress once used twelve calls on one healthy attempt. It now uses five across all 37. Its local gate used to ask when any single signal fired. A 120-step repair chain re-runs its check command throughout, so that signal was normal for that task, not a warning. The attempt it spent its budget on scored 0.978. Asking only on two weak signals together, with a four-turn pause, fixed it. Jev answered “not stuck” all five times, on sessions that were fine. The untrusted-content system was never called in these runs. It only joined when a result came from a web or browser tool, and no task here produced one. It has since been widened to pages fetched through the shell, which agents did in 136 of 577 recorded Terminal-Bench runs. Replayed over 737 of those fetches it flagged 18, and all but one were text really addressed to an AI agent: benchmark task prompts and dataset rows that were themselves system prompts.

A count mismatch found one more bug: the proxy saw 44 calls leave the machine and the ledger held 43. A system that is not awaited can still be running when the session ends. Its answer was dropped with its log line. The answer has to go (it belongs to a session that no longer exists). The line has to stay, because the ledger claims to list every send. Fixed. The fix came from comparing the two counts.

The first sessions long enough to compact

Everything above is a short session. Across every tier and every published run, including Terminal-Bench, no session had ever compacted — so compaction, the system that guides where a compaction cuts, had never been asked anything. A row of zeroes that reads as “did nothing” was really “was never in the room”.

Tier 6 changes that by declaring a 24,000‑token context window instead of a 200,000‑token one. Growing the corpus does not work — an agent with a shell chunks or greps rather than loading it, and a 254,000‑token corpus still produced no compaction at all. A small window overruns on the transcript, which no shortcut avoids.

The guard row is kept as measured and labelled for what it was: this run used the short-lived native guard, and command gating now belongs to the specpi-lancet-guard package, which these runs do not install. The other three rows describe systems the layer still ships.

A hundred and thirty-nine applied verdicts, against zero in every run before it. The layer does act; it had simply never been placed in a situation where it could. Progress also returned tool-error-loop verdicts — the one situation a purpose-built task had entirely failed to produce, which appeared unprompted once sessions ran long. Later replays showed those verdicts were an echo: the question's own state carried the local reasons for asking, and over 174 recorded runs it answered tool-error-loop 624 times out of 624. That finding, and the one below it, withdrew the system.

Where it did not fit: compaction guidance, withdrawn

Then the same tier that finally exercised compaction guidance is the tier that took it out of the layer. On the long task, the arm carrying it solved 8 of 16 against plain SpecPi’s 14 of 16 — and it did so twice, in two runs months apart on two versions, 5/8 against 8/8 and then 3/8 against 6/8. The shorter task showed nothing either time.

Attribution was never in question: compaction was 139 of the 142 verdicts the layer applied over those attempts, with retention applying none and progress three. The mechanism was. The system could not delete anything — it appended one sentence to the summariser’s prompt, with Pi’s own cut point and token budget untouched — so any harm ran through a summary that kept the wrong material. The likeliest explanation is that the sentence told the summariser to prioritise one kind of content, which on a task built so that a fact from the first turn must survive is an instruction to rank that fact below something else. That was never isolated.

It was removed anyway. The difference sits at p = 0.054, which is not significance, and no single test here clears that bar. What decided it was three weak signals agreeing, a clean line of attribution, and the asymmetry of the bet: a system that steers a summary has to earn the risk it takes, and one whose only measurement says it costs solve rate has not. The changelog carries the full comparison, the p‑values and what the removal does not establish. The evaluations page no longer does: the tier suite those runs belong to has been retired in favour of Terminal-Bench 2.0, and its figures are kept in this repository's history rather than deleted, because a measurement that led to a removal is the reason the removal can be defended.

Where it did not fit either: progress detection, withdrawn

Progress detection was withdrawn for a different reason: not a lower solve rate, but no signal. Its local check fired in 174 of 577 recorded Terminal-Bench runs, and those runs did fail more often — 58% passed against 78% of the rest. Jev's verdict added nothing to that. Comparing attempts at the same task, is this session stuck ranked the failures no better than chance (AUC 0.46), and adding the recent commands to its state did not change that. A system that can steer a model has to show it can tell when to, and this one could not.

A check at the end of a run failed the same way before it was built. Most failures here end with the agent stopping on its own, so is the task actually done looked like the best unowned question. Over 479 such runs its answer separated passes from failures only by recognising hard tasks; within a task it scored 0.54–0.58, under a bar of 0.65 set before it ran. Jev answers questions about a piece of content well. Predicting how a session will end from its trace is not one of them.

Both Jev columns cover one harness on one model. The re-run happened separately from the matrix the control comes from, so drift between sittings lands on the layer. The per-system counts avoid that: they are the layer's own record of what it did. scripts/jev-effect.mjs rebuilds every figure above from the stored reports.

Nothing here is on after install. Switching the layer on switches its systems on with it. A layer with every system off runs and does nothing. The choice is saved, so a switch set once stays set. --session limits a change to the current session and leaves the default alone.

• /jev: show the layer state and change it.
• /jev on: the whole layer. Master switch plus every system that is off. Saved for new sessions; /jev on --session for this one only. Nothing under /jev touches the command guard, which is a separate package with its own commands.
• /jev status: current posture, plus each place a key can come from, with the active one marked.
• /jev ledger: local record of every payload sent. Each call is stored in transmissions.jsonl on this machine.
• /jev forget: revoke consent.

In SpecPi Chat the same switches are toggles under package settings. They write the same file the extension reads. Each budget shows the advisor's call count: the total, one row per system, and how many calls changed anything. A bare call count hides “asked six times, changed nothing”. That split is shown instead. Those counts live in a separate usage.json with numbers only, which keeps it safe for a second process to read. Chat reads it and never writes it. The advisor writes it only once the layer is on. Like the settings nearby, the counts belong to the agent folder, not one chat, so each entry carries a date and a running or ended mark.

The default cap is 1536 calls per session, with per-system caps that add up past it, so the total still binds. Treat it as a backstop. A full tier-3 task uses 4 to 7 calls. The busiest attempt on record used 12. Only a loop can reach 1536, so reaching it signals a bug. Cost is not the point (a call runs about $0.00003, so the whole cap is about five cents). The cap limits how much summary leaves the machine and how much waiting a looping run can add. Hitting the cap is announced once per system. Quiet is normal here, so a system gone quiet from an empty budget would otherwise pass for one with nothing to report.

Jev is on OpenRouter, so an OpenRouter key covers the layer. It is resolved the way Pi resolves other provider keys, in Pi's documented order: the openrouter entry from /login openrouter in auth.json, then OPENROUTER_API_KEY from the environment. There is nothing extra to set up. The advisor once read the environment only and reported a missing key for people whose OpenRouter login worked fine. JEV_BACKEND=typesafe selects the TypeSafe direct API instead, which needs a TYPESAFE_API_KEY. The two keys are not interchangeable. The wrong one returns a bare 401.

The command guard needs no key at all: it scores commands locally with LANCET.

Retention, storage, and logging of sent data follow OpenRouter and TypeSafe rules, not SpecPi's. The layer ships off, asks before its first send, caps payload size, and keeps a local record of each one.