v0.3.0 · pre-releaseApache-2.0 modelMIT runtime

LANCET NanoA local classifier that flags risky Bash commands before an agent or a person runs them.

Risky caught
85.8%
351 / 409 benchmark commands
Safe stopped
5.5%
21 / 384 benchmark commands
Parameters
110 M
CodeT5-base encoder
On disk
111 MB
INT8 ONNX
Per command
~22 ms
median, CPU, offline

Results on the 793-command release benchmark

Risky commands caught against safe commands wrongly stopped for LANCET Nano v0.3.0, v0.2.0, v0.1.0, Jev and Laya
Up and to the left is better. Each model was scored once, at its own frozen threshold, on lancet-bench-1: 409 risky and 384 safe commands across 37 tool areas, frozen and screened against all training data before v0.3.0's training data existed.
ROC curves for the four models
How well each model separates risky from safe commands at every threshold. The circles mark the thresholds each model actually uses.
Risky commands caught by tool area
By tool area. v0.3.0 improves most on secrets (24% → 66%) but is weaker on network/remote execution. Counts per area are small, so read these as indicative.
ModelRisky caughtSafe stoppedAUROCSecrets caughtParametersRuns on
LANCET Nano v0.3.085.8%5.5%0.96266%110 MCPU, local, offline
LANCET Nano v0.2.073.6%6.2%0.89624%35 MCPU, local, offline
LANCET Nano v0.1.065.5%5.5%0.86014%35 MCPU, local, offline
Jev (hosted)96.8%7.8%0.98298%undisclosedhosted API
Laya68.5%37.8%0.71848%421 MGPU, local

The benchmark's labels were written by the developer, an AI agent, so this is diagnostic evidence, not independent acceptance. Jev and Laya received a fixed task context; Nano sees only the command.
On the upstream ShellRisk sets (not agent-authored, never trained on), v0.3.0 caught 48.9% vs v0.2.0's 48.3% and stopped 6.3% vs 9.7% of safe commands; on the smaller holdout set it caught fewer (41.9% vs 45.9%).
v0.3.0 was released by owner exception after one overly strict preregistered check (the ShellRisk catch-rate lower bound) failed; the details are in the model card.

Quick start

# 1. verify the download, then the bundle
certutil -hashfile lancet-v0.3.0-nano-cpu-int8.zip SHA256
python verify_bundle.py --strict

# 2. install three pinned CPU dependencies
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt

# 3. classify (JSON lines in, JSON lines out)
$env:OPENBLAS_NUM_THREADS = '1'
'{"command": "terraform destroy -auto-approve", "shell": "bash"}' |
  .\.venv\Scripts\python.exe classify.py --model model
{"score": 0.997, "classification": "risky", "reason": null,
 "experimental": true, "executionAuthorized": false, ...}
RISKY
High confidence. Block or require explicit approval.
REVIEW
Uncertain or unsupported input. Ask a person.
NOT_FLAGGED
No risk found. Not a safety guarantee.
  • Python 3.12 with NumPy, tokenizers and ONNX Runtime CPU. No PyTorch, no GPU, no API key.
  • Keep one process running and stream commands to it; the model loads once.
  • Commands are read as text and never executed.
  • On macOS or Linux, use .venv/bin/python. Those platforms were not benchmarked.
  • If you clone instead of downloading the ZIP: install Git LFS, run git lfs pull, and use bundle/.

How it was trained

v0.3.0 fine-tunes the Salesforce CodeT5-base encoder from its pinned upstream weights, with no earlier LANCET checkpoint, using class-balanced sampling. Its labels come from project authoring and documentation, not from another model's opinions:

  • tldr-pages: each example's one-line description sets the label. "Delete a specific pod" is risky; "List all volumes" is benign.
  • AWS botocore: the operation verb, plus AWS's own sensitive markings on returned fields, so commands that print credentials are risky.
  • Azure CLI, GitHub CLI, kubectl, Docker CLI reference examples, labelled by verb and description, including documented secret reveals such as keys list and --with-decryption.
  • LANCET's project-authored risky/safe pairs, a secrets family, earlier agent-authored evaluation suites (retired into training, original labels), and developer commands from SWE-smith, Terminal-Bench and NL2Bash.
Training sourceRows
Project-authored pairs9,906
tldr-pages examples6,126
Secrets family (project-authored)5,328
Retired evaluation suites4,358
Developer commands (V5 pool)3,054
Azure / GitHub / kubectl / Docker CLI1,428
AWS CLI operations927

No language model, hosted API or human labeler produced any training label.
Development and calibration used tools and families held out from training.
The label rules are heuristic and were not human-adjudicated.

Limits

Credits & licenses

Built on Salesforce CodeT5-base by Yue Wang, Weishi Wang, Shafiq Joty and Steven C. H. Hoi (Apache-2.0).
Training commands come from tldr-pages (CC BY 4.0, © the tldr-pages team and contributors), botocore (Apache-2.0, © Amazon.com, Inc. or its affiliates), the Azure CLI and GitHub CLI (MIT), and kubectl and Docker CLI (Apache-2.0), plus the datasets credited in the notices. None of them endorses LANCET.
Model: Apache-2.0. Runtime: MIT. Datasets are not included or relicensed.
See the NOTICE, the third-party notices and the licenses.