octoturge
564d01816b
`POST /jobs` ended at a `queued` row and an upload URL; nothing moved it. This is the join, plus the two things that join turns out to need. ## The scheduler `src/dispatch` ticks every 5s: for each idle, claimed printer on a farm with a connected gateway, take the oldest unassigned queued job and send `start_print`. A timer rather than an event, because a job becomes dispatchable for two unrelated reasons - someone queued it, or a printer became free - and the second has no event this backend owns. It arrives as a telemetry snapshot, seconds after the fact. A scheduler that only ran on job creation would leave a queue stuck forever behind one busy printer. Claiming is an `UPDATE ... WHERE status = 'queued'` that returns the row, so two ticks cannot both win the same job. Selecting and then updating would let a tick and a nudge each believe they owned it and send two starts. It checks the bucket before dispatching. `POST /jobs` hands back a presigned PUT and returns, so the row exists before the file does; dispatching anyway would have the gateway download R2's 403 error document and give it to a printer as a project. A job whose file never arrives just stays queued, and a zero-byte upload fails outright because it will never become valid. The download URL is presigned at dispatch and not at creation - a queued job may wait days. ## Job state comes from telemetry, not from the gateway's ok This is the part that is not obvious and was learned the expensive way on 2026-10-15: a real print was accepted, uploaded byte-perfect, reached RUNNING, and then the printer paused itself on a physical broken-filament fault. A job marked `printing` on the gateway's reply would have claimed to be printing for the rest of the day. So `advanceJobsFrom` runs on every telemetry snapshot and the mapping lives in `job_status.ts`, apart from the database so every pair can be tested without a Postgres or a printer. The line that matters: an idle printer does *not* complete a job that is still `uploading`. A printer is idle for the entire transfer - minutes of it, at the 0.2 MiB/s measured over FTPS - so reading that as completion would mark every job successful seconds after dispatch, before the file reached the machine. 15 tests cover the table exhaustively. Because the gateway now acks a start immediately (it must - awaiting the transfer would stop its telemetry for minutes, and telemetry is what this depends on), a failure after the ack has no result message to travel on. Hence the reaper: a job `uploading` for 20 minutes is failed, not requeued, because the printer may actually be printing it and a requeue would put a second job on a working machine. ## Unclaimed printers are visible now `persistTelemetry` still refuses to create printer rows for gateway ids it does not recognise - a typo in a `printers.toml` becoming a permanent phantom printer is worse. But the refusal was silent, which made the worst first-run experience in the platform: gateway connects, reports happily, fleet stays empty, and the only evidence is in stderr. The observation is now kept and served by `GET /farms/:id/unclaimed-printers`, with `gatewayOnline` alongside it because an empty list otherwise cannot distinguish "all claimed" from "no gateway ever connected". Claiming is still `POST /printers`, still owner-only. 77 tests pass.
continuum-backend
Cloud control plane for Continuum. Bun + ElysiaJS, PostgreSQL (via pgCat/PgBouncer, with pgvector for frame embeddings), Valkey Streams, and Cloudflare R2 storage.
Run
bun install
scripts/dev-env.sh up # Valkey + Postgres/pgvector, and print the URLs
scripts/dev-env.sh schema # apply drizzle/ - same as bun run db:migrate
cp .env.example .env # fill in the URLs it printed, plus R2_*, SUPABASE_JWT_SECRET
bun run dev
scripts/dev-env.sh uses docker-compose.yml when a Docker daemon is
answering, and falls back to the same two services from apt when there is no
usable container runtime - some sandboxes cannot run Docker at all, because
dockerd needs CAP_NET_ADMIN for its bridge and user namespaces to unpack an
image, and neither can be granted from inside. scripts/dev-env.sh status
says which backend is in use.
Schema changes go through migrations: edit src/db/schema/, run
bun run db:generate, and commit the file it writes to drizzle/ along with
the schema change. scripts/dev-env.sh push reshapes a local database straight
from the schema with no migration file, which is faster while a table is still
moving but records nothing and will drop a column to make things match - so
generate before committing. drizzle/0000_*.sql creates the vector extension
itself; that line is hand-added, since drizzle-kit does not emit extension
statements and the migration would otherwise only apply to a database somebody
had already prepared.
Test
bun test # unit + protocol contract tests, no services needed
scripts/e2e-edge.sh # the whole edge path: real database, real printers
scripts/e2e-edge.sh seeds a farm, claims its printers, mints an edge token,
runs the real continuum-proxy against this backend, and asserts that live
telemetry reached Postgres and that a command reached a printer. It only ever
sends a fan speed, and only to a printer reporting IDLE.
scripts/capture-gateway-frames.js is the smaller tool: it stands in for this
backend and records what a gateway sends, for refreshing the fixtures in
tests/fixtures/.
Layout
src/db/- Drizzle schema (printers,farms,print_jobs,frame_embeddings) and client, pointed at pgCat.src/db/validators.tsderives Elysia/TypeBox request validators straight from the Drizzle tables viadrizzle-typebox, so the Postgres schema and the API validators come from one definition instead of two hand-synced ones.src/s3/- R2 presigned URL helpers (PUTfor client uploads,GETfor the edge proxy).src/queue/- Valkey client (ioredis) +XADDhelper pushing ontostream:camera_framesforcontinuum-ai-worker.
Valkey, not Redis: Redis 7.4+ is RSALv2/SSPL and 8.0+ adds AGPLv3, none of
which a commercial product can ship. Valkey is BSD-3-Clause and speaks the
same RESP protocol, so ioredis is unchanged - only the server differs. See
continuum-common/docs/LICENSES.md. docker-compose.yml here brings up
Valkey and Postgres+pgvector for local development.
src/routes/-/auth,/farms,/printers,/jobs, and the edge-gateway WebSocket at/ws/edge/v1. That route's wire format is not defined here: it isschemas/continuum/v1/edge/in continuum-schemas, validated through its TypeBox mirror in@continuum/ts-types, because both ends of it are separate programs in different languages and they had already drifted apart once.printers.gateway_printer_idis what maps a gateway's own printer ids ("p1p") onto rows here; telemetry for an unclaimed id is logged and skipped, never inserted.
frame_embeddings.embedding requires the vector extension. The first
migration creates it, and scripts/dev-env.sh up also does so for the push
path, which does not run migrations.
Dependency pins
elysia, @sinclair/typebox and the drizzle-* packages are pinned tightly
on purpose, not out of caution. Caret ranges here resolved into a combination
that could not boot: elysia: ^1.1.26 installed 1.4.30, which calls
t.Module and so needs TypeBox >=0.34, while drizzle-typebox: ^0.2.1
installed a version whose peers demand drizzle-orm >=0.36 against the
^0.33.0 pinned here. Typechecking passed throughout - only starting the
server failed. tests/dependency_identity.test.ts guards the part of this that
can silently come back.