FRAMESHIFT

Frameshift (formerly Elite Trader) is a companion application for the space simulator Elite Dangerous. It reads the game's on-disk journal files in real time, maintains a local market database seeded from a community galaxy dump and kept fresh by a live data firehose, and serves a web cockpit to any browser on the LAN — most importantly a tablet mounted next to the HOTAS, running the touch-first flight panel mode.

Design philosophy

• Local-first. No account, no API key, no cloud backend. Every feature works from data on the player's own disk. Trade routes are computed locally against a 1.5 GB SQLite database rather than by calling a third-party website.
• Panel-first. The tablet cockpit is the primary interface; the desktop layout is the secondary view. UI decisions are made for touch at arm's length.
• Honest data. Prices carry confidence chips and conservative ranges built from age and order-book depth. The app labels what it estimates rather than presenting guesses as facts.
• Anti-bloat. A tab must justify itself with a self-contained multi-card workflow; features that duplicate mature external tools are declined. The rail is capped at ~12 icons.
• Teach the game. UI copy explains mechanics in plain terms (engineering rolls, exobiology bonuses, neutron plotting) — the target user includes brand-new commanders.

Technology stack

Language

Python 3.12 (backend) · native ES2022 modules checked in strict TypeScript mode, no framework (frontend)

Server

Flask 3 behind Werkzeug's threaded WSGI server, LAN-bound

Storage

SQLite (WAL mode) — two databases plus JSON stores; no external services

Desktop shell

pywebview window hosting the same web UI; also runs fully headless

Realtime feed

ZeroMQ subscriber on the EDDN community relay

Packaging

PyInstaller one-file Windows executable, built and checksummed by GitHub Actions

Dependencies

flask, pywebview, requests, pyzmq, pydirectinput (Windows-only) — all version-pinned

One Python process sits between three worlds: the game's files and input queue on the same machine, the community network (EDDN, Spansh, GitHub), and the player's devices on the LAN. Everything the process learns converges on one thread-safe snapshot (AppState) and two SQLite databases; everything the player sees is a view over those.

Three properties fall out of this shape and are enforced throughout the codebase:

• The game is read-only, except autoplot. Frameshift never writes to game files. The single write path into the game is the autoplot keystroke sequence (§09), which is explicit, user-triggered and abortable.
• The internet is optional. With networking removed, journal telemetry, the local trading engine, ops boards and analytics keep working. Network-backed features (Spansh planners, EDDN freshness, updates) degrade with explicit UI messaging.
• Devices are untrusted by default. Every request passes the security ladder in §06 — including requests from the desktop's own window, which are simply granted the loopback rung.

app.py is the composition root. It parses flags (--headless, --port), refuses to double-start (a live /api/state probe on the target port opens a window to the existing instance instead), then wires the runtime in a fixed order so that every consumer finds its producer already alive:

Thread inventory

Cross-thread discipline is simple and consistent: every shared structure is owned by exactly one module. AppState guards all mutation behind one lock and hands out deep-copied snapshots; SQLite connections are opened per-operation (or thread-locally shared inside a reconstruction session — §04); the seed importer and the EDDN listener coordinate a database swap with an explicit pause/resume handshake so a mid-import commodity message can never write into a half-built file.

Elite Dangerous appends one JSON event per line to a session journal and rewrites small status files in place. Frameshift's entire knowledge of the player comes from replaying and tailing these files — there is no game API. The pipeline has two phases with one invariant: ingestion is idempotent, so bootstrap and live tailing may overlap, crash and re-run without double counting.

Bootstrap file selection

The walk-back starts at the newest journal and keeps adding older files until it has at least BOOTSTRAP_MIN_FILES (12) sessions and every context boundary is satisfied — the most recent LoadGame, unsold exobiology samples (a SellOrganicData/Died boundary), and colonisation-depot context that spans sessions — capped at BOOTSTRAP_MAX_FILES (25). Replaying ~12 files every start is therefore by design: it is how session-spanning state (massacre stacks, unsold samples, carrier context) survives a restart without a hidden cache that could drift from the truth.

Commander attribution

Every event is attributed to a stable local profile key derived from the journal's commander identity (commander_id — never a network identity). Stub sessions that never reach Commander/LoadGame (menu-only launches) are marked processed but not ingested, and per-file bookkeeping tables never migrate during profile repair — the two rules that keep the "unassigned history" counter honest.

Two SQLite databases with sharply different lifecycles. market.db is rebuildable — a cache of the galaxy that can be re-seeded from scratch at any time. commander.db is durable — the player's own history, which nothing can regenerate. The code treats them with matching levels of care: the market DB is swapped wholesale under a lock; the commander DB is migrated in place with transactional, key-checked table rebuilds.

The event ledger

eventledger.py is the durable counterpart to the ephemeral AppState: every useful journal event is stored once as canonical JSON compressed with zlib, keyed by a content digest. This is what makes features retroactive — when a new analytics view or workflow ships, a replay over the ledger rebuilds its derived tables from the player's full history instead of starting empty. The compressed payloads also make the bootstrap/live overlap idempotent by construction.

Other stores

security.json

Paired-device registry — token digests only, atomic writes (§06)

settings store

User settings with validated keys (settings.py)

data/tts/

Piper binary + voice models, SHA-256 pinned downloads

data/extensions/

Declarative extension packs, fingerprint-approved (§08)

data/webview/

Desktop window profile — pywebview runs with private_mode=False so localStorage survives

data/backups/

Automatic pre-migration copies of commander.db

The Flask server exposes 89 routes under /api/, plus the static UI. Authorization is default-deny: a route added tomorrow gets a scope requirement automatically — unlisted GETs require READ, unlisted mutations require CONTROL, and only two endpoints (pairing itself) are public. Long-running search POSTs that mutate nothing are explicitly allow-listed back down to read.

Commander ownership is route metadata, not an informal client convention. 46 routes always require an X-Frameshift-Commander identity and one extension-test route requires it conditionally when the request reads journal history. The server captures one state snapshot when the request begins, rejects a missing or stale identity, and never falls back to a mutable active profile during a journal handoff. A policy audit enumerates every route and fails when ownership metadata is missing.

Error handling is uniform: routes return error_response() bodies built from UserFacingError subclasses (ValidationError, NotFoundError). Exception text is only echoed to a client when a developer explicitly marked the message as written for the player — an unexpected exception can never leak internals into a response (CWE-209 by construction).

Frameshift is its own edge server on a home LAN, reachable by anything on the network and — via the browser — by any website a LAN user visits. The trust model is deliberately account-free: the desktop is trusted because it is local; everything else earns a capability. Every request climbs the same ladder:

Hardening conventions

• Path confinement. Every filesystem sink that touches a user-influenced path normalizes and then executes inside a positive startswith(root) branch, keeping the trust boundary explicit for review, static analysis and regression tests.
• No command injection surface. The game launcher runs a fixed Steam/Frontier command; user input never reaches a shell.
• Supply-chain pinning. Runtime dependencies are version-pinned; GitHub Actions are pinned to commit SHAs; TTS binaries and voice models are verified against hard-coded SHA-256 digests before use.
• Update integrity. A release download must match the published size, carry an MZ header and match its published SHA-256 before the swap script is ever written (§10).
• Privacy-safe telemetry out. EDDN uploads are built per-schema from explicit field lists — commander identity never leaves the machine (§08).

The browser application is 134 native ES modules (20,895 physical lines) behind a 21-line HTML shell. It remains framework-free and zero-build: Flask serves the source graph directly and PyInstaller packages it recursively. There are no runtime packages, generated bundles, CDN assets or transpilation step; npm is used only for pinned development gates. The same application state renders two layouts:

Flight panel mode — default

A touch cockpit for a tablet: a 100 px icon rail navigates 12 pages (STATUS, TRADE, MARKET, EXPLORE, GUIDES, STATS, ENG, GALAXY, OPS, ROLES, LOCAL, SETTINGS), a sticky status strip shows system/station/destination with fuel and cargo minibars on every page, and swipe gestures page the rail. Cards get corner brackets, a boot splash, and optional CRT ambience (off by default — ambient motion is opt-in). Touch targets are ≥46 px; number inputs render in the mono face.

Desktop mode

The same cards in an 11-tab layout for keyboard-and-mouse use, hosted either in any browser or in the pywebview desktop window.

Module and state architecture

Composition

src/main.js is a bootstrap-only entrypoint. Named modules under bootstrap/ mount views, wire controls, coordinate state and start polling; they do not own feature DOM selectors or listeners.

Application state

core/store.js owns one typed ApplicationState snapshot. Features subscribe to complete snapshots and keep mutable workspace state in narrow feature-owned modules rather than ambient globals.

Transport

Twelve named domain clients import explicit request/response contracts. Only core/http.js may call fetch(); presentation code cannot embed API paths or import the transport directly.

Commander handoff

Scoped requests capture the stable commander ID plus store generation, send X-Frameshift-Commander, abort on profile change, and reject stale responses before feature state can be committed.

Safe rendering

Escaping html`…` templates and one reviewed render() boundary own dynamic markup. Direct HTML sinks are prohibited; runtime values use escaped templates, text nodes or typed DOM helpers.

Feature ownership

features/ owns domain behavior, features/views/ owns static panes, features/controls/ owns idempotent listener wiring, and shell/ owns the desktop/Panel presentations.

CSS graph

Twenty-six split sheets plus the reusable Holo Button component load through one entrypoint with declared layers: reset → tokens → base → components → features → panel → themes → motion. Every split sheet stays under 400 lines.

Performance

The shipped graph is 134 modules and 462 relative import edges with zero cycles. CI budgets module count, total JS/CSS/HTML bytes and the largest individual module for tablet startup.

EDDN — inbound eddn.py

A ZeroMQ subscriber on the community relay (tcp://eddn.edcd.io:9500) applies commodity/3 messages to market.db for any station whose system the seed knows — zlib-inflated, schema-checked, fleet carriers and legacy-galaxy clients filtered. Net effect: local prices are minutes old, not days, without Frameshift ever asking a website for a price.

EDDN — outbound eddn_upload.py

Frameshift also gives back. Docked market snapshots are contributed by default; broader journal/snapshot schemas are an explicit opt-in. Every message is built for a concrete schema from allow-listed fields rather than forwarding raw game JSON — commander-specific fields cannot leak, and location augmentation fails closed. Uploads are stamped with the real gameversion/gamebuild captured from the journal header.

Spansh spansh.py

Two very different uses: the one-time galaxy dump seed (~4 GB download, ~15 min import — §04), and per-request planner calls (neutron route, Road-to-Riches, exobiology bodies, station/service search, per-system dumps for community bio signals). Named reference systems are primary; known coordinates are retried when Spansh doesn't index a fresh discovery. All calls carry timeouts and surface as user-facing errors, never tracebacks.

GitHub Releases updater.py

Hourly update checks against the Releases API; the download/verify/swap dance is described in §10. The legacy repository name 301-redirects, and every release publishes assets under both the old and new executable names — an install from any historical version can still update forward.

Piper TTS tts.py

Neural voice callouts run entirely locally: a pinned Piper build and voice model (6-voice catalog, SHA-256 verified) are downloaded on demand; one persistent child process synthesizes phrases at ~0.3× real-time with a WAV cache, so every paired device hears the same voice with no cloud round-trip.

Extension host extensions.py

Extension packs are declarative by default: a manifest that can inspect journal events and emit alerts or objective suggestions, but cannot execute code — safe to install by copying a directory. An optional process adapter exists for advanced integrations and stays disabled until the user approves the pack's exact content fingerprint; the child receives a minimal JSON document on stdin and has no API access. A capability boundary, not an OS sandbox — and documented as exactly that.

The feature that makes every system name in the UI a button: ◎ PLOT types a route into the game itself. Elite Dangerous has no API for this, so autoplot.py drives the galaxy map with synthetic keystrokes (pydirectinput), reading the player's own keybindings (bindings.py) rather than assuming defaults.

The sequence — open map, focus search, type the name, commit, zoom, plot — takes several seconds, so it is engineered to be safe to trigger from a tablet and safe to abort: the requesting control turns into a CANCEL button, /api/plot returns 409 when the game isn't running, and success is verified rather than assumed by watching NavRoute.json for the expected destination. A voice callout confirms the plotted route.

Releases are cut by pushing a v* tag. Nothing is built by hand, and the pipeline will not produce an executable from a failing gate. The tag build repeats strict frontend checks, 113 Vitest cases, 16 Chromium/WebKit browser executions and all 55 isolated Python test scripts before it stamps the version. After packaging, the executable must serve a coherent recursive ESM graph with the expected cache and version headers before GitHub receives an asset:

Layered test suite 87 files · 3 runtimes

Each test layer owns the behavior it can observe best. Python exercises backend state and persistence, Vitest imports the real ESM modules against a lightweight DOM, and Playwright verifies complete desktop and tablet journeys in actual browser engines.

The Python runner parses each script, records every local test_* definition and fails if the entrypoint did not actually call one. This preserves the application's required process isolation without permitting silent test definitions.

Risk-shaped coverage

• Journal interpretation — event handlers, bootstrap selection, recovery, rebuild progress, commander scoping and profile-repair regressions (the "48 unassigned rows" class of bug gets a permanent test).
• Data integrity — ledger idempotency, migration rebuilds, EDDN schema validation against the official schemas, reseed swap safety.
• Security and identity — pairing, scopes, host/origin checks, path confinement, updater verification, profile activation coherence and the 46-required/one-conditional commander-route policy.
• UI behavior — source modules are imported directly. Store generation, scoped requests, feature handoffs, escaping, focus behavior, reduced motion and the specialist/tablet paths are asserted as behavior rather than source-text tokens.

Static and release gates

• Architecture — strict TypeScript checkJs, explicit API contracts, module-size limits, no legacy bridges or type exemptions, and no presentation-layer transport or unsafe DOM seam bypasses.
• Graph integrity — all 462 relative imports resolve, follow the permitted layer direction and form zero cycles; bare runtime package imports are rejected.
• UI budgets — DOM IDs, CSS layers/import order, module count, JS/CSS/HTML weight, formatting and lint rules are deterministic CI failures rather than review suggestions.
• Browser matrix — Chromium runs the complete journey suite; WebKit runs the tagged tablet path. Fixed locale, time zone, reduced motion and retained traces keep failures reproducible.
• Packaged smoke — the frozen executable must recursively serve its same-origin ESM graph with one stamped version, no-cache UI assets and no-store API state.
• CI matrix — every push/PR compiles Python and runs frontend, browser and isolated-Python gates on Windows and Ubuntu; the release job repeats them before building.
• Profiling culture — performance work starts with py-spy on the live process (it attaches to the frozen exe), not with guesses; the two big wins (idle CPU ~51%→0%, cold start 6 min→39 s) both came from profiler evidence overturning the obvious suspects.

The backend's 44 modules contain 20,240 physical lines concentrated around journal interpretation, HTTP policy and local data. The frontend is intentionally distributed: 20,895 lines across 134 bounded modules rather than one application file.

Frontend ownership map

Repository map

frameshift/
├── app.py                  composition root (headless or windowed)
├── elite/                  backend package — 44 modules (table above)
│   └── data/               bundled engineering catalog (gzip)
├── ui/                     frontend, served from disk — no build step
│   ├── index.html          21-line document shell
│   ├── panel-bootstrap.js  synchronous pre-paint Panel guard
│   ├── hb.js               Holo Button interaction helper
│   ├── src/
│   │   ├── main.js         ESM entrypoint
│   │   ├── core/ · data/
│   │   ├── api/contracts/
│   │   ├── features/       controls/ · views/ · domain modules
│   │   └── shell/ · bootstrap/
│   ├── styles/             layered CSS graph and single index.css
│   └── fonts/              self-hosted Chakra Petch + IBM Plex Mono
├── tests/
│   ├── test_*.py           55 isolated backend scripts
│   ├── unit/               Vitest modules (31 files total, incl. colocated)
│   └── e2e/                Chromium + WebKit tablet journeys
├── tools/                  architecture/CSS/DOM/performance gates,
│                           isolated Python runner, served-graph verifier
├── docs/                   living UI contract, completed plan, extension docs
├── package.json            pinned frontend development and test tooling
├── packaging/              Windows version-resource template
├── .github/workflows/      test.yml (push/PR, 2 OSes) · release.yml (tags)
└── data/ (runtime)         market.db · commander.db · security.json
                            backups/ · tts/ · extensions/ · webview/

Repository

github.com/TannerMidd/frameshift — source, releases, issues

Showcase

tannermidd.github.io/frameshift — feature tour with live screenshots

User docs

Project wiki — getting started, flight panel, trading, autoplot, pairing, security and troubleshooting

UI architecture

Living contribution contract — module ownership, request identity, safe rendering and enforced gates

Modernization record

Completed implementation plan — final decisions and acceptance criteria delivered in v2.5.0

Downloads

Latest release — Windows one-file executable + SHA-256

This dossier

Refreshed from the v2.5.0 source tree; figures and counts are measured, not estimated