v0.4.3 · pre-releaseApache-2.0 modelMIT runtimeBash · PowerShell · cmd

LANCET NanoA local classifier that flags risky shell commands before an agent or a person runs them.

Triage Score
68.3
three benchmarks · v0.4.2: 54.4
Risky caught
77.0%
asked or blocked · three benchmarks
Safe stopped
9.0%
asked or blocked · three benchmarks
On disk
115 MB
111 M params · INT8 ONNX
Per command
~14 ms
median, CPU, offline

Triage Score: command guards at their shipped settings

Triage Score at shipped settings across 7,911 commands from three benchmarks: LANCET Nano v0.4.3 68.3, LANCET Nano v0.4.2 54.4, LANCET Nano v0.4.0 47.1, LANCET Nano v0.4.1 44.9, LANCET Nano v0.3.0 44.7, LANCET Nano v0.2.0 40.7, LANCET Nano v0.1.0 40.5, Jev 38.9, Kestrel 30.7, ModernBERT bash 23.2, bash-classify 14.3, Laya 13.3, bev-decider 13.3, sh-guard 10.9
Every risky command asked about or blocked earns a point; allowed earns nothing. The score shrinks in proportion when more than 10% of safe commands are stopped. Three benchmarks (7,911 commands), weighted by size. Scores use the new lancet-bench-2-next and are lower for every guard than on the retired lancet-bench-1; see why.

LANCET Nano vs other command guards

LANCET Nano v0.4.3 vs 13 other command guards, Triage Score at shipped settings on 7,911 commands (risky caught, safe stopped): LANCET Nano v0.4.3 68.3 (77.0%, 9.0%), Jev 38.9 (90.0%, 23.4%), verdict-shell-safety 32.4 (56.3%, 18.0%), Kestrel 30.7 (68.1%, 37.1%), ModernBERT bash 23.2 (48.1%, 22.8%), dcg 19.8 (19.8%, 6.9%), AutoShell-0.8B 19.2 (98.8%, 70.9%), Gyra 17.9 (28.2%, 13.4%), bash-classify 14.3 (92.9%, 65.8%), Laya 13.3 (87.3%, 69.6%), bev-decider 13.3 (94.4%, 75.3%), laya-cli-gate 12.5 (71.2%, 57.5%), sh-guard 10.9 (97.9%, 90.5%), Shieldstral-1.0-3B 6.1 (6.1%, 0.2%)
LANCET Nano v0.4.3 against every other command guard we have tested, each scored once at its own shipped setting on the same 7,911 commands. Every other guard either stops more than 10% of safe commands or catches fewer than a third of risky ones; Nano catches 77.0% of risky commands while stopping 9.0% of safe ones.

v0.4.3 vs v0.4.2

LANCET Nano v0.4.3 vs v0.4.2: Triage Score per benchmark and caught, stopped and blocked rates
v0.4.3 is newly trained on a new curriculum, reads long commands in full and adds PowerShell and cmd, at the same size and speed. On lancet-bench-2-next it catches the same 78.0% of risky commands as v0.4.2 while stopping 12.4% of safe ones instead of 20.0%; on ShellRisk-Bench it catches more (64.8% vs 60.1%). The neutral set is its one Triage loss (85.8% caught vs 91.0%, with fewer safe commands stopped).

Where v0.4.3 gains and loses

v0.4.3 vs v0.4.2 on the same commands: Triage points gained and lost by benchmark, caught and interrupted rates per test set, and command-by-command disagreements
Both releases on exactly the same commands. Left of centre: commands v0.4.2 handled and v0.4.3 does not; right: the reverse.

Triage Score by release

Triage Score by release on lancet-bench-2-next: v0.1.0 40.5, v0.2.0 40.7, v0.3.0 44.7, v0.4.0 47.1, v0.4.1 44.9, v0.4.2 54.4, v0.4.3 68.3
Every LANCET Nano release scored the same way, at its shipped setting, on the same three benchmarks. Grey ticks: each release's score on the retired lancet-bench-1. v0.4.3 was trained on lancet-bench-1, so it has no tick.

Triage Score across three benchmarks

Triage Score by guard, split into points from lancet-bench-2-next, ShellRisk-Bench and a neutral set: LANCET Nano v0.4.3 68.3, LANCET Nano v0.4.2 54.4, LANCET Nano v0.4.0 47.1, LANCET Nano v0.4.1 44.9, LANCET Nano v0.3.0 44.7, LANCET Nano v0.2.0 40.7, LANCET Nano v0.1.0 40.5, Jev 38.9, Kestrel 30.7, ModernBERT bash 23.2, bash-classify 14.3, Laya 13.3, bev-decider 13.3, sh-guard 10.9
Each bar is one guard's Triage Score on 7,911 commands; the colours show how many points it earned on each benchmark: lancet-bench-2-next (3,204 commands, up to 58 points), the ShellRisk-Bench test split (4,194, up to 20) and a neutral set of outside-party commands (513, up to 21). A benchmark's points are its weight (58% / 20% / 21%, by size) times the guard's score on it: every risky command asked about or blocked earns a point, and the score shrinks in proportion when more than 10% of safe commands are stopped. Each guard is scored once at its shipped setting. Hover a segment for that benchmark's own score.
GuardTriage Scorelancet-bench-2-nextShellRisk testNeutral setRisky caughtSafe stoppedParametersRuns on
LANCET Nano v0.4.368.363.164.885.877.0%9.0%111 MCPU, local
LANCET Nano v0.4.254.439.060.191.077.1%13.7%110 MCPU, local
LANCET Nano v0.4.047.141.160.650.855.7%10.5%110 MCPU, local
LANCET Nano v0.4.144.938.170.539.367.9%14.6%110 MCPU, local
LANCET Nano v0.3.044.744.346.643.655.3%11.6%110 MCPU, local
LANCET Nano v0.2.040.739.347.737.955.2%13.6%35 MCPU, local
LANCET Nano v0.1.040.534.945.651.058.4%14.1%35 MCPU, local
Jev38.943.632.432.190.0%23.4%undisclosedhosted API
verdict-shell-safety32.427.633.344.956.3%18.0%151 MCPU, local
Kestrel30.712.7100.014.068.1%37.1%—local
ModernBERT bash23.216.240.925.648.1%22.8%150 Mlocal
dcg19.826.44.116.519.8%6.9%ruleslocal
AutoShell-0.8B19.210.222.441.098.8%70.9%0.8 BGPU, local
Gyra17.919.04.127.828.2%13.4%421 M + rulesGPU, local
bash-classify14.313.516.814.092.9%65.8%ruleslocal
Laya13.311.016.217.187.3%69.6%421 MGPU, local
bev-decider13.310.818.714.894.4%75.3%0.4 BGPU, local
laya-cli-gate12.511.812.314.571.2%57.5%421 MGPU, local
sh-guard10.910.212.111.797.9%90.5%ruleslocal
Shieldstral-1.0-3B6.14.17.89.96.1%0.2%3 BGPU, local

Jev received task context; the others see only the command. Per-benchmark columns are each benchmark's own Triage Score. Guards that support only Bash answer the 138 PowerShell and cmd commands as "ask".

Triage Score by area on lancet-bench-2-next for Nano v0.4.2, v0.4.3, Jev, Kestrel and Laya
Detail by area on lancet-bench-2-next. v0.4.3 gains most on shell semantics, language runtimes and dev tools, Windows and infrastructure as code, and gives back a few points on cloud and Kubernetes commands, where v0.4.2 was already strong. Kestrel and Laya stop most of the safe look-alikes in nearly every area.

Why every score dropped: a harder benchmark

Until v0.4.2 the Triage Score used lancet-bench-1 (793 Bash commands). The best guards were close to its ceiling (Jev caught 96.8% of its risky commands, v0.4.2 92.4%), and it was narrow: Bash only, short single-line commands, a quarter of them about secrets.

From v0.4.3 the Triage Score uses lancet-bench-2-next, a new, much more comprehensive benchmark that no LANCET release was trained, tuned or calibrated on. Because it is much harder, every guard scores lower than before, LANCET and comparators alike: v0.4.2 scored 82.7 under the previous Triage Score and 54.4 under the new one. The model did not change; the test did. Scores are only comparable on the same benchmark.

  • 3,204 commands built as 1,602 risky/safe twins. Every risky command has a safe look-alike that differs only in the detail that decides the risk, such as reading a secret's expiry date instead of its value.
  • 289 scenarios in 40 areas: cloud CLIs, Kubernetes, databases, Git and CI, containers, credentials, filesystems and backups, infrastructure as code, deceptive previews, encoded input, shell quoting, expansion, redirection and control flow, language runtimes, messaging, remote execution, macOS and Windows administration.
  • Composition: every scenario is repeated inside a subshell, a function, a command substitution, a nested bash -c, a pipeline and a background job.
  • Three shells: 3,066 Bash, 102 PowerShell and 36 cmd commands; 1,803 are multi-line (lancet-bench-1 had none).
  • lancet-bench-1 is retired, now part of LANCET's training data, and stays published. lancet-bench-2-next is kept private so it stays an unseen test.

Quick start

# 1. verify the download, then the bundle
certutil -hashfile lancet-v0.4.3-nano-cpu-int8.zip SHA256
python verify_bundle.py --strict

# 2. install three pinned CPU dependencies
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt

# 3. classify (JSON lines in, JSON lines out)
$env:OPENBLAS_NUM_THREADS = '1'
'{"command": "terraform destroy -auto-approve", "shell": "bash"}' |
  .\.venv\Scripts\python.exe classify.py --model model
'{"command": "Stop-Computer -Force", "shell": "powershell"}' |
  .\.venv\Scripts\python.exe classify.py --model model
{"score": 0.992, "classification": "risky", "reason": null,
 "experimental": true, "executionAuthorized": false, ...}
{"score": 0.989, "classification": "review", "reason": "uncertainty-band",
 "experimental": true, "executionAuthorized": false, ...}
RISKY
High confidence. Block or require explicit approval.
REVIEW
Uncertain or unsupported input. Ask a person.
NOT_FLAGGED
No risk found. Not a safety guarantee.
  • Python 3.12 with NumPy, tokenizers and ONNX Runtime CPU. No PyTorch, no GPU, no API key.
  • shell is bash, powershell or cmd.
  • Keep one process running and stream commands to it; the model loads once.
  • Commands are read as text and never executed. Long commands are read in full, one 512-token window at a time.
  • On macOS or Linux, use .venv/bin/python. Those platforms were not benchmarked.
  • If you clone instead of downloading the ZIP: install Git LFS, run git lfs pull, and use bundle/.

How it was trained

v0.4.3 is the Salesforce CodeT5+ 220M encoder, fine-tuned from its pinned upstream weights with no earlier LANCET checkpoint, plus a small pooling head that reads long commands as overlapping windows. Training ran to convergence (cosine learning-rate decay to zero). Its labels come from project authoring, documentation and source datasets, not from another model's opinions:

  • Command-semantics curriculum (new): project-authored risky/safe twins for Bash, PowerShell and cmd, including long multi-line scripts.
  • tldr-pages: each example's one-line description sets the label. "Delete a specific pod" is risky; "List all volumes" is benign.
  • AWS botocore: the operation verb, plus AWS's own sensitive markings on returned fields, so commands that print credentials are risky.
  • Azure CLI, GitHub CLI, kubectl, Docker CLI reference examples, labelled by verb and description.
  • ShellRisk-Bench training split: attack-technique commands from Atomic Red Team and InternalAllTheThings, and everyday developer commands from SWE-smith and Terminal-Bench, with their upstream labels.
  • Shell Safety v2 training split (MIT): agent-session shell commands with their author's allow/deny labels.
  • LANCET's project-authored risky/safe pairs, contrast sets, a secrets family, earlier evaluation suites and the retired lancet-bench-1 (original labels).
Training sourceRows
Command-semantics curriculum (new)21,544
Project-authored pairs9,184
Secrets family (project-authored)7,728
tldr-pages examples6,093
ShellRisk-Bench training split5,951
Earlier evaluation suites5,047
Developer commands (V5 pool)2,956
Contrast sets and shell-escape atlas2,598
Shell Safety v2 training split2,475
Azure / GitHub / kubectl / Docker CLI1,418
Retired benchmarks (lancet-bench-1, draft)1,300
AWS CLI operations917

No language model, hosted API or human labeler produced any training label.
Development and calibration used tools and families held out from training.

Limits

Credits & licenses

Built on Salesforce CodeT5+ 220M by Yue Wang, Hung Le, Akhilesh Deepak Gotmare, Nghi D. Q. Bui, Junnan Li and Steven C. H. Hoi (BSD-3-Clause).
Training commands come from tldr-pages (CC BY 4.0, © the tldr-pages team and contributors), botocore (Apache-2.0, © Amazon.com, Inc. or its affiliates), the Azure CLI and GitHub CLI (MIT), kubectl and Docker CLI (Apache-2.0), and ShellRisk-Bench by Kontext Security, including Atomic Red Team (MIT, © Red Canary) and InternalAllTheThings (no license declared), and Shell Safety v2 by tomngdev (MIT), plus the datasets credited in the notices. None of them endorses LANCET.
Model: Apache-2.0. Runtime: MIT. Datasets are not included or relicensed.
See the NOTICE, the third-party notices and the licenses.