fcde6f51550191d73e5d77bd4bc41ad0c4046c32
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 octoturge
|
818f28ea5d
|
Pin the discovery frame against a real capture
The gateway and backend halves of `discovered_printers` are hand-synced, so every hand-written example in this file agrees with whichever side wrote it - which is exactly how the two ends of /ws/edge/v1 drifted into different protocols before. A capture is the only thing that catches that. This one came off a live LAN with a Bambu P1P and H2C on it, through the gateway's own serde enum, with its fleet file pointed at an empty file so nothing was filtered out. The addresses and the shape are as captured. The two serials are placeholders of the captured shape. They identify someone's physical hardware, no test depends on their value, and no other fixture here carries one - so the real ones are not something to commit on the owner's behalf. |
||
|
octoturge
|
cd7aed0028
|
Receive and serve a gateway's printer discovery results
The gateway can find printers on its own network and USB ports; nothing carried that back. So adding a printer in the console still began with someone walking to a machine to read its IP address off the screen. Three pieces: - A `discovered_printers` handler on /ws/edge/v1, recording the latest scan per farm. - `discoveredByFarm`, one entry per farm, replaced wholesale by each report. - `GET /farms/:id/discovered-printers`, viewer-gated like its unclaimed sibling. Why a separate store rather than folding this into `unclaimedByFarm`: an unclaimed printer is already in the gateway's `printers.toml`, so someone has written down its address and credential and the gateway is talking to it - all that is missing is the row. A discovered printer was merely seen. Nobody configured it, no credential for it exists anywhere, and the gateway cannot reach it as a printer. One is a row waiting to be created; the other is a suggestion. Keyed per farm rather than per printer because discovery has no stable key to merge on: a Bambu printer has a serial, a Klipper host has only the address DHCP gave it this week, and merging on that accumulates a phantom candidate per lease. The route cross-checks candidates against the farm's existing printer rows so the console can grey one out instead of offering a duplicate - the gateway filters its own fleet file but knows nothing about rows registered in the cloud and never claimed. `matchedOn` reports the basis because the two bases are not equally sure: a serial identifies a machine, an address identifies a DHCP lease and may well be last month's printer. `discovery: null` means the gateway has never reported a scan, which is the normal case because discovery is off by default in the gateway's config. That is deliberately distinguishable from a report whose `printers` is empty, which means it looked and found nothing. The protocol tests pin the parts that matter for safety: an over-long name is refused, and so is a candidate carrying an `access_code`. That second one is not tidiness - no probe can learn an access code, so a candidate that has one got it elsewhere, and the closed object is what stops this message quietly becoming a channel for secrets nobody meant to send to the cloud. |
||
|
octoturge
|
0d87c8cbce
|
Make the edge protocol contract test pass on its own
tests/edge_protocol.test.ts validates schemas carrying format: "uuid" and format: "uri" but registered neither. TypeBox treats an unregistered format as a failure, not a pass, so every start_print in that file was rejected - and the file still went green, because some other test in the same run imported Elysia first and registered the formats globally. Run alone it failed. A contract test that depends on which files ran before it is the worst way for one to behave. It now imports src/validation/formats, which exists for exactly this and gains a "uri" validator: anything the platform parses as an absolute URL. Lenient on purpose - the only user is a presigned object-store GET that this backend mints itself, so a stricter pattern would have to anticipate whatever query parameters a signer adds, and rejecting a URL R2 would have honoured costs a print for no security. |
||
|
octoturge
|
57af710b1a
|
Verify the edge token at the upgrade, not in the first frame
`/ws/edge/v1` took its credential from a `token` field on the `hello` message. It now takes it from `Authorization: Bearer` on the upgrade request, verified before the socket exists, so an unauthenticated peer gets an HTTP 401 and no WebSocket: no entry in `connections`, no frame ever parsed, no state of any kind allocated for it. The old arrangement worked, and every message type refused to act without a verified farm. But "it cannot do anything" is a weaker guarantee than "it does not exist" - an anonymous peer still got a live socket and had its JSON run through `EdgeClientMessageSchema` and the message switch before proving anything, and it took a timeout to stop anyone holding sockets open for free. A `?token=` query parameter was never on the table: a bearer credential in a URL lands in access logs, proxy logs and error breadcrumbs, which is the thing the proxy's hand-written `Debug` for `PrinterCommand` already exists to prevent. The decision lives in `src/auth/edge_upgrade.ts`, a leaf module with no imports but the error types - the same reason as `dispatch/job_status.ts` and `stream.ts`. This is the authentication boundary for the entire gateway protocol, so it should be the last thing only reachable through a running database. It checks `scope` and not just the signature (the same secret could be made to sign something else one day), rejects a token whose farm claim is absent or not a string (that would otherwise produce a connection whose `farmId` is `undefined`, which reads as "every farm" to a lookup), and answers every failure with one message - which part of a forged token to fix next is not something to tell its holder. `conn.farmId` is therefore set from the verified token in `open`, and the `hello` handler no longer verifies anything. It still checks `message.farm_id` against the token's farm, so a gateway naming a farm its token was not minted for is refused rather than silently working against the wrong one. The handshake timer stays but changed meaning: it is no longer an auth timeout, just a bound on an authenticated socket that never identifies its gateway. Verified, not assumed. That a throwing `resolve` fails the upgrade is a framework behaviour the whole design rests on, so `tests/ edge_upgrade.test.ts` stands up a real Elysia app with a real JWT plugin and asserts a socket never opens without a valid edge token, and that a resolved value reaches `ws.data`. The real `continuum-proxy` was also run against this composition: valid token gives HTTP 101 with the farm taken from the token, forged token gives 401 at the handshake. The committed captures predate this and still contain `token: "dev-token"`. That validates - the schema is a liberal reader - and is harmless, because the field is now ignored and such a gateway is refused at the upgrade for having no header. A test says so. 95 tests (was 85), `bunx tsc --noEmit` clean. |
||
|
octoturge
|
564d01816b
|
Dispatch queued jobs, and derive their state from the printers
`POST /jobs` ended at a `queued` row and an upload URL; nothing moved it. This is the join, plus the two things that join turns out to need. ## The scheduler `src/dispatch` ticks every 5s: for each idle, claimed printer on a farm with a connected gateway, take the oldest unassigned queued job and send `start_print`. A timer rather than an event, because a job becomes dispatchable for two unrelated reasons - someone queued it, or a printer became free - and the second has no event this backend owns. It arrives as a telemetry snapshot, seconds after the fact. A scheduler that only ran on job creation would leave a queue stuck forever behind one busy printer. Claiming is an `UPDATE ... WHERE status = 'queued'` that returns the row, so two ticks cannot both win the same job. Selecting and then updating would let a tick and a nudge each believe they owned it and send two starts. It checks the bucket before dispatching. `POST /jobs` hands back a presigned PUT and returns, so the row exists before the file does; dispatching anyway would have the gateway download R2's 403 error document and give it to a printer as a project. A job whose file never arrives just stays queued, and a zero-byte upload fails outright because it will never become valid. The download URL is presigned at dispatch and not at creation - a queued job may wait days. ## Job state comes from telemetry, not from the gateway's ok This is the part that is not obvious and was learned the expensive way on 2026-10-15: a real print was accepted, uploaded byte-perfect, reached RUNNING, and then the printer paused itself on a physical broken-filament fault. A job marked `printing` on the gateway's reply would have claimed to be printing for the rest of the day. So `advanceJobsFrom` runs on every telemetry snapshot and the mapping lives in `job_status.ts`, apart from the database so every pair can be tested without a Postgres or a printer. The line that matters: an idle printer does *not* complete a job that is still `uploading`. A printer is idle for the entire transfer - minutes of it, at the 0.2 MiB/s measured over FTPS - so reading that as completion would mark every job successful seconds after dispatch, before the file reached the machine. 15 tests cover the table exhaustively. Because the gateway now acks a start immediately (it must - awaiting the transfer would stop its telemetry for minutes, and telemetry is what this depends on), a failure after the ack has no result message to travel on. Hence the reaper: a job `uploading` for 20 minutes is failed, not requeued, because the printer may actually be printing it and a requeue would put a second job on a working machine. ## Unclaimed printers are visible now `persistTelemetry` still refuses to create printer rows for gateway ids it does not recognise - a typo in a `printers.toml` becoming a permanent phantom printer is worse. But the refusal was silent, which made the worst first-run experience in the platform: gateway connects, reports happily, fleet stays empty, and the only evidence is in stderr. The observation is now kept and served by `GET /farms/:id/unclaimed-printers`, with `gatewayOnline` alongside it because an empty list otherwise cannot distinguish "all claimed" from "no gateway ever connected". Claiming is still `POST /printers`, still owner-only. 77 tests pass. |
||
|
octoturge
|
b5382ab47c
|
Accept a telemetry stage, and prove the field is really there
`stage` comes from the shared schema, so this is contract tests only.
The test that matters asserts a *wrongly typed* stage is rejected rather than
only that a valid one is accepted. `PrinterTelemetrySchema` permits unknown
properties - verified empirically, `{ totally_bogus: 1 }` validates - so
`{ stage: 54 }` passes whether or not the field is declared, and a test built
only on the positive case would keep passing if someone deleted it.
Worth knowing separately: that open `additionalProperties` means a gateway
sending a misspelled field name is accepted and the value silently dropped,
which is the same drift this suite exists to catch. Not changed here, because
tightening it needs a check against what real gateways send.
|
||
|
octoturge
|
398a739edd
|
Expose control_print, and say what the gateway's ok does not mean
The route's body is `PrinterCommandSchema` straight from the shared schema, so adding the variant there is what exposes it here - that is the point of taking the type from one place. What needed saying is in the comments: stop is a cancel with no undo, and a gateway returning ok means the printer accepted the command rather than that the print state changed, since an idle P1P answers success to all three verbs. A UI that puts stop where pause is expected, or that treats the 200 as confirmation, gets this wrong. The contract test gains the three control frames and one guard: a verb only one vendor recognises - `cancel`, Moonraker's word for it - must not validate, or it reaches a gateway that cannot map it. Mirrored in continuum-proxy/tests/uplink_protocol.rs. |
||
|
octoturge
|
42d4b3820a
|
Add a runnable dev environment, and fix the pins that stopped it booting
The edge route had never been run. Adding a way to run it turned up that the server could not start at all, for two independent dependency reasons that typechecking could not see: - `elysia: ^1.1.26` had resolved to 1.4.30, which calls `t.Module` - a TypeBox 0.34 API - while this repo pinned TypeBox 0.33. `t.Module is not a function`. - `drizzle-typebox: ^0.2.1` resolved to a version whose peers demand `drizzle-orm >= 0.36`, against the `^0.33.0` pinned here. Importing it threw `Export named 'getViewSelectedFields' not found`. Both are caret ranges that drifted into combinations nobody chose. elysia, @sinclair/typebox and the drizzle packages are now pinned tightly, TypeBox forward to 0.34.52 (which satisfies Elysia and drizzle-typebox 0.1.1 alike), and tests/dependency_identity.test.ts fails if the TypeBox version ever differs between here and @continuum/ts-types - the skew that made every shared schema invalid, with an error message pointing at Standard Schema instead. Dropping to drizzle-typebox 0.1.1 widens `jsonb` in the generated validators, so `metadata` needs an explicit object shape in validators.ts; the column is declared `$type<Record<string, unknown>>()` and a validated body otherwise is not assignable to `.values()`. scripts/: - `install-docker.sh` - one-time host setup, kept separate because it touches the machine rather than containers. - `dev-env.sh up|down|reset|status|psql|schema|env` - Valkey + Postgres/pgvector. Uses docker-compose.yml when a daemon answers, and otherwise runs the same two services from apt. That fallback is not hypothetical: Docker cannot run in this sandbox at all. dockerd needs CAP_NET_ADMIN to build its bridge; with `--bridge=none --iptables=false` it starts but cannot unpack an image without CAP_SYS_ADMIN; with `--storage-driver=vfs` it still fails because registering a layer needs user namespaces, which are blocked outright. All three are host-level grants, so no script can work around them - hence a second path rather than a broken first one. - `e2e-edge.sh` - the test the unit tests cannot be. Seeds a farm, claims its printers, mints an edge token, runs the real continuum-proxy, and asserts live telemetry reached Postgres and a command reached a printer. It only sends a fan speed, only to a printer reporting IDLE, and puts it back afterwards. - `capture-gateway-frames.js` - records what a gateway really sends, for refreshing tests/fixtures/. Read-only unless FAN_TEST_PRINTER is set. The test comment referred to this before it existed. Verified end to end, from a dropped and freshly migrated database: `h2c` persisted as printing/RUNNING 22% layer 2/25 nozzle 230 bed 35 with 3 HMS alerts, `p1p` as idle/IDLE, and `POST /printers/:id/commands` reached the P1P's fan and came back ok:true. An unclaimed printer is refused 409 PRINTER_UNREACHABLE rather than guessed at. Two things the e2e script got wrong first, both now fixed in it: it waited for the first printer row instead of all of them, which silently tested only the v2 printer (a v1 Bambu reports one tick later); and `$!` after `( cd ... ) &` is the subshell, so cleanup left an orphaned gateway reconnecting with a token for a deleted farm and polluting the next run. Note: `drizzle/` is gitignored, so the generated migration is not committed and there is no migration history in the repo. Left as-is deliberately - that is a workflow call, flagged in PLAN.md. |
||
|
octoturge
|
e2c1e0abe2
|
Speak the gateway's actual protocol on /ws/edge/v1, and persist telemetry
This route and continuum-proxy had drifted into entirely different protocols. It expected an `auth` handshake the gateway never sends, one printer per telemetry message where the gateway sends a batch, UUID printer ids a gateway has no way to know, and camelCase field names against the gateway's snake_case. It had no way to send a command to a printer at all. Telemetry was acknowledged and thrown away - the handler said as much. It now implements schemas/continuum/v1/edge/ via its TypeBox mirror in @continuum/ts-types, so both ends mirror one written contract instead of each other's assumptions. The missing link was identity: a gateway knows its printers as "p1p", this database knows them as UUIDs, and nothing mapped between them. printers.gateway_printer_id is that mapping, unique per farm since two farms may each have a printer called "p1p". An unrecognised id is logged and skipped rather than inserted - a gateway should not be able to create printer records by mentioning them, or a typo in someone's printers.toml becomes a permanent phantom printer. Telemetry now lands in `printers` as a latest-value snapshot, extending the pattern status/nozzle_temp_c/bed_temp_c already used: target temperatures, progress, layers, remaining minutes, job name, active alert count. Also vendor_state alongside the narrowed status enum, because a state we do not recognise yet would otherwise be flattened and lost - and an unrecognised state deliberately leaves status untouched rather than falling back to "idle", since idle is what the scheduler reads as safe to send work to. Commands can now go down: sendPrinterCommand() takes our own printer id, translates it on the way out, and waits for the gateway's result, with a 10s timeout and a registry keyed by farm. Pending commands are failed immediately when a gateway disconnects rather than making every caller wait out the timeout for a socket that is already gone. POST /printers/:id/commands exposes it, validated against the same shared schema the gateway accepts, so this route cannot offer something no gateway can execute - which notably means there is still no way to start a print. tests/edge_protocol.test.ts is the contract test, run against real frames captured from a live gateway talking to a real P1P and H2C rather than hand-written examples - hand-written ones agree with whoever wrote them, which is how this drifted in the first place. The second fixture is the same capture from before the gateway was taught to omit empty fields, kept deliberately: the validator must keep accepting explicit nulls. Not yet applied to a database: this adds columns but no migration, and the repo has none committed. Run db:generate (or push) before the next deploy. |