7a0e8ba6ef054cc638787ce4c44a4557fce56cf2
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 octoturge
|
7a0e8ba6ef
|
Close an edge socket that never authenticates
The edge protocol carries its token in the first message rather than the handshake, which is the right call for a WebSocket - a browser cannot set headers, and a query parameter puts a bearer credential into every access and proxy log. But the cost of that choice is a window in which an open socket belongs to nobody, and nothing was closing it. Every other message type already refuses to act without a `farmId`, so such a socket could do nothing. It did sit in `connections` forever though, and that map is keyed by whoever can reach the port. Ten seconds, which is generous for a round trip a real gateway makes immediately on open; a tighter bound would only drop gateways on slow links. The timer is cleared after the token verifies and the farm matches, so a client retrying a bad token on one socket still gets dropped rather than being handed an unbounded number of attempts. 85 tests, `bunx tsc --noEmit` clean. |
||
|
octoturge
|
4de0d86d43
|
Tell a client where to watch a printer's camera
`continuum-proxy` now generates go2rtc's configuration from the camera
URLs the printers themselves report, and sends two new things up: a
`stream_base_url` on its hello, and a `camera_stream` per printer in
telemetry. This is the other end of both.
`GET /printers/:id/stream` answers `{ streamUrl, cameraId }`, which is
everything a browser needs to open a WebRTC session against the gateway's
go2rtc. 404 rather than 200-with-nulls when either half is missing: a
client handed `{ streamUrl: null }` has to remember to check, and one that
forgets renders a player that can never load. The two causes get
different messages, because they are different problems for whoever is
debugging - a P1/A1 that can never stream is not a gateway that happens
to be offline - and the printer is checked first so a disconnected
gateway cannot mask the permanent case.
The resolution lives in `src/stream.ts` with no imports at all, for the
same reason `dispatch/job_status.ts` does: `../db` opens a connection at
import time, and these two messages are exactly the part worth testing
without one.
Storage, both decided against the obvious alternative:
- `stream_base_url` lives on the in-memory connection, so it goes away
with the socket. It is a LAN address that is only reachable while that
gateway is up; a column on `farms` would keep handing out an address
nothing answers after the gateway disconnected or moved.
- `camera_stream` gets a plain column, because every sibling telemetry
field already has one, and it is cleared when absent like the rest - a
printer that loses its stream must stop being offered one.
`cameraStream` is also added to `printerProjection`, which that file
deliberately requires someone to decide per column. It is a go2rtc stream
name, which is our own printer id echoed back, not a credential, and
having it in the list a client already fetches is what saves it from
calling the stream endpoint once per printer to discover that most of
them 404 by design.
Worth writing down, and it is in the route's comment too: `streamUrl` is
a LAN address and nothing relays it. A browser on the gateway's network
can play the stream and one outside cannot, and this endpoint cannot tell
which the caller is. Good enough for an operator in the workshop, a real
limit for anyone else, and a relay is the fix that does not exist yet.
85 tests (was 77), `bunx tsc --noEmit` clean.
|
||
|
octoturge
|
564d01816b
|
Dispatch queued jobs, and derive their state from the printers
`POST /jobs` ended at a `queued` row and an upload URL; nothing moved it. This is the join, plus the two things that join turns out to need. ## The scheduler `src/dispatch` ticks every 5s: for each idle, claimed printer on a farm with a connected gateway, take the oldest unassigned queued job and send `start_print`. A timer rather than an event, because a job becomes dispatchable for two unrelated reasons - someone queued it, or a printer became free - and the second has no event this backend owns. It arrives as a telemetry snapshot, seconds after the fact. A scheduler that only ran on job creation would leave a queue stuck forever behind one busy printer. Claiming is an `UPDATE ... WHERE status = 'queued'` that returns the row, so two ticks cannot both win the same job. Selecting and then updating would let a tick and a nudge each believe they owned it and send two starts. It checks the bucket before dispatching. `POST /jobs` hands back a presigned PUT and returns, so the row exists before the file does; dispatching anyway would have the gateway download R2's 403 error document and give it to a printer as a project. A job whose file never arrives just stays queued, and a zero-byte upload fails outright because it will never become valid. The download URL is presigned at dispatch and not at creation - a queued job may wait days. ## Job state comes from telemetry, not from the gateway's ok This is the part that is not obvious and was learned the expensive way on 2026-10-15: a real print was accepted, uploaded byte-perfect, reached RUNNING, and then the printer paused itself on a physical broken-filament fault. A job marked `printing` on the gateway's reply would have claimed to be printing for the rest of the day. So `advanceJobsFrom` runs on every telemetry snapshot and the mapping lives in `job_status.ts`, apart from the database so every pair can be tested without a Postgres or a printer. The line that matters: an idle printer does *not* complete a job that is still `uploading`. A printer is idle for the entire transfer - minutes of it, at the 0.2 MiB/s measured over FTPS - so reading that as completion would mark every job successful seconds after dispatch, before the file reached the machine. 15 tests cover the table exhaustively. Because the gateway now acks a start immediately (it must - awaiting the transfer would stop its telemetry for minutes, and telemetry is what this depends on), a failure after the ack has no result message to travel on. Hence the reaper: a job `uploading` for 20 minutes is failed, not requeued, because the printer may actually be printing it and a requeue would put a second job on a working machine. ## Unclaimed printers are visible now `persistTelemetry` still refuses to create printer rows for gateway ids it does not recognise - a typo in a `printers.toml` becoming a permanent phantom printer is worse. But the refusal was silent, which made the worst first-run experience in the platform: gateway connects, reports happily, fleet stays empty, and the only evidence is in stderr. The observation is now kept and served by `GET /farms/:id/unclaimed-printers`, with `gatewayOnline` alongside it because an empty list otherwise cannot distinguish "all claimed" from "no gateway ever connected". Claiming is still `POST /printers`, still owner-only. 77 tests pass. |
||
|
octoturge
|
1db96caa91
|
Stop falling back to a committed edge signing secret
`EDGE_JWT_SECRET` was read as `process.env.EDGE_JWT_SECRET ?? "dev-edge-secret-change-me"` in two places. Anything holding that string can mint an edge JWT for any `farm_id` it likes and open `/ws/edge/v1` as that farm's gateway: enough to feed a farm false telemetry and to receive the commands meant for its printers. It is in a git repository, so it is not a secret. The `??` made it worse rather than better. It only catches null and undefined, and the committed `.env.example` ships `EDGE_JWT_SECRET=` *empty* - so a deployment that copied the template and filled in everything else signed its edge JWTs with the empty string and never reached the fallback at all. That is exactly the case the fallback existed to cover. Both reads now come from `src/auth/edge_secret.ts`, which throws at module load if the variable is unset or empty. Same shape as `verifier.ts` and for the same reason: a backend that boots and then fails every gateway handshake looks like a network problem somewhere else. There is no development default on purpose - a secret that works out of the box is a secret that reaches production, and `openssl rand -hex 32` is a one-time cost. Tests: the empty-string case, since that is the one the template produced, and a grep for both shapes that reintroduce the hole - a literal passed to `jwt()`, and a defaulted read of the variable - plus an assertion that the variable is read in exactly one place. 60 pass. |
||
|
octoturge
|
398a739edd
|
Expose control_print, and say what the gateway's ok does not mean
The route's body is `PrinterCommandSchema` straight from the shared schema, so adding the variant there is what exposes it here - that is the point of taking the type from one place. What needed saying is in the comments: stop is a cancel with no undo, and a gateway returning ok means the printer accepted the command rather than that the print state changed, since an idle P1P answers success to all three verbs. A UI that puts stop where pause is expected, or that treats the 200 as confirmation, gets this wrong. The contract test gains the three control frames and one guard: a verb only one vendor recognises - `cancel`, Moonraker's word for it - must not validate, or it reaches a gateway that cannot map it. Mirrored in continuum-proxy/tests/uplink_protocol.rs. |
||
|
octoturge
|
b42cbac299
|
Fix the frame producer's field names, and pipeline the batch
The other end of the `stream:camera_frames` drift. This side wrote `farmId`, `printerId`, `printJobId`, `r2ObjectKey` and `capturedAt`; `continuum-ai-worker` read `printer_id` and nothing else. Every task was delivered and acknowledged, and every frame was attributed to a printer called "unknown", with no error on either side - a stream entry is a flat string map, so there is nothing to parse and nothing to fail. The field names are now snake_case, from `schemas/continuum/v1/ai/frame_task.schema.json`. That is the convention this repo already follows for anything shared with a Rust consumer (see `/ws/edge/v1`), and it is not an inconsistency with the camelCase HTTP routes: this is not an HTTP route. `print_job_id` is omitted from the XADD when there is no job rather than written as `""`. An empty string is a value: a consumer that believes it looks up job id `""`, or puts it in a uuid column. A stream entry has no null, so "no value" has to mean "no field". The producer checks each task against the shared schema before writing it. The check is not for the database's sake - a stream accepts any field name - it is to make this exact class of drift loud. It earned its keep immediately by failing on every task with `Unknown format 'uuid'`: TypeBox ships no format validators and treats an *unregistered* format as a failure rather than skipping it, so a schema using `format: "uuid"` checked outside Elysia rejects everything. `src/validation/formats.ts` registers the two this codebase uses, guarded by `Has` so Elysia's own registrations still win for the route validators. (The HTTP routes were never affected - importing Elysia registers ~24 formats including `uuid`, which I confirmed by asserting a non-uuid body gets a 422.) A `frame_batch` now goes onto the stream in one pipelined round trip instead of one per frame, which was a round trip each while holding up the WebSocket message handler - and a farm produces roughly a frame per printer per second. ioredis reports per-command pipeline errors in the result tuple rather than throwing, so those are checked explicitly; a pipeline where one XADD failed otherwise looks like success. `frameTaskToStreamFields` and the compiled check live in their own module because `src/queue/index.ts` opens a Valkey connection on import. Keeping the wire format pure means `tests/frame_task_contract.test.ts` asserts it with nothing running. Tests, mirroring the worker's own so neither side can agree only with itself: the exact key set, values carried through unchanged, `print_job_id` present when there is a job and the key absent when there is not, no camelCase key ever emitted, and the uuid and date-time formats actually enforced - that last one fails if the format registration is lost (everything rejected) or becomes a no-op (nothing rejected). Plus `PrintAlertSchema`, which this repo will have to parse once the fan-out route exists. `scripts/e2e-ai-frames.sh` runs this producer against the real worker binary over real Valkey and asserts the field names by reading them back out of Valkey - rather than from either side's source, which is what made the original bug invisible. Two faults in the first draft of that script, both worth naming because a test that passes for the wrong reason is worse than no test: it stranded *every* entry on a dead consumer, so the ordinary delivery path was never exercised at all; and it grepped the subscriber output for "print_alert", which matched `redis-cli`'s own subscription confirmation line and reported a published alert that did not exist. |
||
|
octoturge
|
d8f8653dee
|
Authenticate and authorize every route: org layer + Supabase JWKS
Until now every HTTP route was unauthenticated. A probe against the
running server returned a farm's `edgeProxyToken` and a printer's
`accessCode` to an anonymous caller, and that pair is enough to connect
as a farm's gateway and drive its printers. The previous commit stopped
the credentials appearing in responses; this one makes the routes ask
who is calling.
Authentication. Access tokens are verified against the Supabase
project's published JWKS via `createSupabaseJwtVerifier` from
`@continuum/ts-core`: RS256, public key only, no secret in this
process. It replaces the HS256 `@elysiajs/jwt` path, where
`SUPABASE_JWT_SECRET` was simultaneously the verification key and a
key that could mint tokens, present in the environment of every
instance. `src/auth/verifier.ts` builds it at module load so a
misconfigured deployment fails at boot rather than on first login.
The edge-gateway token stays symmetric and that is not an
inconsistency: this service is both its issuer and its only verifier,
so there is no second party to keep a secret from. It now carries an
`exp`, which it did not, despite the response saying
`expiresInSeconds: 3600`.
Tenancy. An org layer above farms - `organisations`,
`org_members(org_id, user_id, role)`, `farms.org_id` - so a farm can
have more than one operator without sharing an account. `farms.owner_id`
is gone. Authorization is one rule applied everywhere: resolve the
thing being acted on back to an organisation and check the caller's
role there, user -> org -> farm -> printer, with nothing in between
carrying permissions of its own. The ladder is
`viewer` < `operator` < `owner`; printer commands and job submission
need `operator`, because they move physical machines, and that is the
one boundary here that is not about data.
A non-member gets 404, not 403: a 403 would confirm that a given farm
or printer id exists and belongs to someone, which is an enumeration
oracle over the whole system. 403 is kept for a member who may see the
record but not do this to it.
List endpoints scope in the `where` clause, not after the query - a
filter applied to the result is one `return` away from being skipped.
They are also bounded now (`limit`/`offset`, default 50, max 200
refused rather than clamped).
`ownerId` and `submittedBy` no longer come from request bodies. Nor do
the printer telemetry columns: `POST /printers` is narrowed to the
columns a client owns, because a client that could POST
`status: "idle"` could make the scheduler send work to a printer that
is mid-print.
Also fills the CRUD gaps that made the API unusable - PATCH and DELETE
for farms, printers, jobs and orgs, and membership management - plus
one error shape for the whole API via `toErrorResponse`, so an
authorization failure is a throw that leaves the handler rather than a
status somebody has to remember to set.
Guards on the two changes that cannot be undone through the API:
removing or demoting an organisation's last owner, and deleting an
organisation while farms are still attached (it would cascade to every
printer, print job and stored frame).
Migration 0001 is hand-edited after generation. drizzle-kit emitted
`ADD COLUMN org_id uuid NOT NULL`, which cannot apply to a populated
table, and dropped `owner_id` without moving what it encodes anywhere.
It now adds the column nullable, creates one organisation per distinct
legacy owner with that user as its `owner` member, points each farm at
it, and only then tightens the column and drops `owner_id`. Verified
both ways: from empty via `drizzle-kit migrate`, and against a seeded
database where two farms shared an owner and a third did not.
Testing. `scripts/dev-auth.ts` is a local RS256 issuer publishing a
JWKS at the path the backend fetches, so the real verification path is
testable without a Supabase project. The alternative was a test-only
bypass in the backend - an env var making it trust an unsigned token -
which would be a permanent hole one misconfiguration from production,
and would mean the code under test is not the code that ships.
- `tests/authz.test.ts` (19 tests): the role ladder written out in full
rather than derived the same way the implementation derives it, and
the real JWKS path - a token signed by a key outside the JWKS, an
expired token and a token with `aud: "anon"` (Supabase's own anon key
is a valid project JWT) are all rejected, with the same message, so
probing cannot tell which part of a forged token to fix next.
- `scripts/e2e-authz.sh` (68 assertions, real Postgres, real routes):
four users across two organisations; anonymous 401s, cross-org 404s,
viewer-cannot-command, operator-cannot-rotate, a spoofed
`submittedBy` in the body ignored, no credential in any read, and the
rotated token still minting an edge JWT.
- `scripts/e2e-edge.sh` updated for the org layer and now authenticates
as a seeded `operator` - the weakest role that should work, so the
run would fail if the command boundary were not where it claims to
be. Re-verified against both live printers: telemetry from `h2c` and
`p1p` persisted, and an authenticated `set_fan_speed` went
HTTP -> WebSocket -> MQTT -> P1P and back `ok:true`, fan returned
to 0.
Two bugs found in the test scripts while writing this, both the same
shape as the earlier `pkill -f` incident: `kill "${PID:-0}"` with an
unset PID signals the whole process group, including the harness, and
`uuidgen` is not installed in this sandbox.
|
||
|
octoturge
|
01bcc73996
|
Stop returning credentials in API responses
`GET /printers` returned `accessCode` and `POST /farms` returned
`edgeProxyToken`, both to callers with no authentication of any kind. Verified
against a running server, not inferred:
POST /farms (no auth) -> "edgeProxyToken": "a39c1709-..."
GET /printers (no auth) -> {'accessCode': 'SECRET123', ...}
Those are not incidental fields. `accessCode` is the printer's own LAN
password, which Bambu uses for MQTT and as the FTPS password. `edgeProxyToken`
trades at POST /auth/edge-token for an edge JWT, which is enough to connect to
/ws/edge/v1 as that farm's gateway and send commands to its printers - a path
this repo's own e2e script walks end to end. So reading one unauthenticated
list was enough to take over a farm's machines.
src/db/projections.ts now defines, per table, which columns may appear in a
response, and the routes select those instead of `*`. Opt-in rather than a
deny-list: a column added later is excluded until somebody decides it should
be public, which is the safe direction to be wrong in. One helper builds both
shapes Drizzle needs - the `{ col: true }` map for the relational query API and
the column references for `.returning()` - from a single list of names, so the
two cannot drift.
`POST /farms/:id/edge-proxy-token` gives that credential one deliberate home.
It rotates and returns the new value rather than revealing the stored one: a
reveal endpoint hands the same secret to everyone who asks and leaves no trace,
whereas rotating makes a leaked token useless and reprovisioning the fix. The
cost is that it locks out a gateway still using the old token, which is the
right trade for a credential that drives physical machines.
This is defence in depth and NOT the fix. Every route here is still
unauthenticated and nothing scopes a farm to a caller - `ownerId` and
`submittedBy` are still taken from the request body, which means a client
declares who owns what. That is the next piece of work and it needs a decision
about the tenancy model first.
|
||
|
octoturge
|
42d4b3820a
|
Add a runnable dev environment, and fix the pins that stopped it booting
The edge route had never been run. Adding a way to run it turned up that the server could not start at all, for two independent dependency reasons that typechecking could not see: - `elysia: ^1.1.26` had resolved to 1.4.30, which calls `t.Module` - a TypeBox 0.34 API - while this repo pinned TypeBox 0.33. `t.Module is not a function`. - `drizzle-typebox: ^0.2.1` resolved to a version whose peers demand `drizzle-orm >= 0.36`, against the `^0.33.0` pinned here. Importing it threw `Export named 'getViewSelectedFields' not found`. Both are caret ranges that drifted into combinations nobody chose. elysia, @sinclair/typebox and the drizzle packages are now pinned tightly, TypeBox forward to 0.34.52 (which satisfies Elysia and drizzle-typebox 0.1.1 alike), and tests/dependency_identity.test.ts fails if the TypeBox version ever differs between here and @continuum/ts-types - the skew that made every shared schema invalid, with an error message pointing at Standard Schema instead. Dropping to drizzle-typebox 0.1.1 widens `jsonb` in the generated validators, so `metadata` needs an explicit object shape in validators.ts; the column is declared `$type<Record<string, unknown>>()` and a validated body otherwise is not assignable to `.values()`. scripts/: - `install-docker.sh` - one-time host setup, kept separate because it touches the machine rather than containers. - `dev-env.sh up|down|reset|status|psql|schema|env` - Valkey + Postgres/pgvector. Uses docker-compose.yml when a daemon answers, and otherwise runs the same two services from apt. That fallback is not hypothetical: Docker cannot run in this sandbox at all. dockerd needs CAP_NET_ADMIN to build its bridge; with `--bridge=none --iptables=false` it starts but cannot unpack an image without CAP_SYS_ADMIN; with `--storage-driver=vfs` it still fails because registering a layer needs user namespaces, which are blocked outright. All three are host-level grants, so no script can work around them - hence a second path rather than a broken first one. - `e2e-edge.sh` - the test the unit tests cannot be. Seeds a farm, claims its printers, mints an edge token, runs the real continuum-proxy, and asserts live telemetry reached Postgres and a command reached a printer. It only sends a fan speed, only to a printer reporting IDLE, and puts it back afterwards. - `capture-gateway-frames.js` - records what a gateway really sends, for refreshing tests/fixtures/. Read-only unless FAN_TEST_PRINTER is set. The test comment referred to this before it existed. Verified end to end, from a dropped and freshly migrated database: `h2c` persisted as printing/RUNNING 22% layer 2/25 nozzle 230 bed 35 with 3 HMS alerts, `p1p` as idle/IDLE, and `POST /printers/:id/commands` reached the P1P's fan and came back ok:true. An unclaimed printer is refused 409 PRINTER_UNREACHABLE rather than guessed at. Two things the e2e script got wrong first, both now fixed in it: it waited for the first printer row instead of all of them, which silently tested only the v2 printer (a v1 Bambu reports one tick later); and `$!` after `( cd ... ) &` is the subshell, so cleanup left an orphaned gateway reconnecting with a token for a deleted farm and polluting the next run. Note: `drizzle/` is gitignored, so the generated migration is not committed and there is no migration history in the repo. Left as-is deliberately - that is a workflow call, flagged in PLAN.md. |
||
|
octoturge
|
e2c1e0abe2
|
Speak the gateway's actual protocol on /ws/edge/v1, and persist telemetry
This route and continuum-proxy had drifted into entirely different protocols. It expected an `auth` handshake the gateway never sends, one printer per telemetry message where the gateway sends a batch, UUID printer ids a gateway has no way to know, and camelCase field names against the gateway's snake_case. It had no way to send a command to a printer at all. Telemetry was acknowledged and thrown away - the handler said as much. It now implements schemas/continuum/v1/edge/ via its TypeBox mirror in @continuum/ts-types, so both ends mirror one written contract instead of each other's assumptions. The missing link was identity: a gateway knows its printers as "p1p", this database knows them as UUIDs, and nothing mapped between them. printers.gateway_printer_id is that mapping, unique per farm since two farms may each have a printer called "p1p". An unrecognised id is logged and skipped rather than inserted - a gateway should not be able to create printer records by mentioning them, or a typo in someone's printers.toml becomes a permanent phantom printer. Telemetry now lands in `printers` as a latest-value snapshot, extending the pattern status/nozzle_temp_c/bed_temp_c already used: target temperatures, progress, layers, remaining minutes, job name, active alert count. Also vendor_state alongside the narrowed status enum, because a state we do not recognise yet would otherwise be flattened and lost - and an unrecognised state deliberately leaves status untouched rather than falling back to "idle", since idle is what the scheduler reads as safe to send work to. Commands can now go down: sendPrinterCommand() takes our own printer id, translates it on the way out, and waits for the gateway's result, with a 10s timeout and a registry keyed by farm. Pending commands are failed immediately when a gateway disconnects rather than making every caller wait out the timeout for a socket that is already gone. POST /printers/:id/commands exposes it, validated against the same shared schema the gateway accepts, so this route cannot offer something no gateway can execute - which notably means there is still no way to start a print. tests/edge_protocol.test.ts is the contract test, run against real frames captured from a live gateway talking to a real P1P and H2C rather than hand-written examples - hand-written ones agree with whoever wrote them, which is how this drifted in the first place. The second fixture is the same capture from before the gateway was taught to omit empty fields, kept deliberately: the validator must keep accepting explicit nulls. Not yet applied to a database: this adds columns but no migration, and the repo has none committed. Run db:generate (or push) before the next deploy. |
||
|
octoturge
|
6d9db5adc1
|
Replace em dashes with hyphens across docs and comments | ||
|
octoturge
|
3f0a51194b
|
Derive Elysia request validators from Drizzle tables via drizzle-typebox | ||
|
octoturge
|
fad0a52690
|
Initial boilerplate scaffold for continuum-backend
Bun + ElysiaJS control plane: Drizzle/Postgres schema (printers, farms, print_jobs, pgvector frame_embeddings), R2 presigned URLs, Redis Streams frame ingestion queue, REST routes, and the /ws/edge/v1 edge-gateway WebSocket uplink. |