Dispatch queued jobs, and derive their state from the printers
`POST /jobs` ended at a `queued` row and an upload URL; nothing moved it. This is the join, plus the two things that join turns out to need. ## The scheduler `src/dispatch` ticks every 5s: for each idle, claimed printer on a farm with a connected gateway, take the oldest unassigned queued job and send `start_print`. A timer rather than an event, because a job becomes dispatchable for two unrelated reasons - someone queued it, or a printer became free - and the second has no event this backend owns. It arrives as a telemetry snapshot, seconds after the fact. A scheduler that only ran on job creation would leave a queue stuck forever behind one busy printer. Claiming is an `UPDATE ... WHERE status = 'queued'` that returns the row, so two ticks cannot both win the same job. Selecting and then updating would let a tick and a nudge each believe they owned it and send two starts. It checks the bucket before dispatching. `POST /jobs` hands back a presigned PUT and returns, so the row exists before the file does; dispatching anyway would have the gateway download R2's 403 error document and give it to a printer as a project. A job whose file never arrives just stays queued, and a zero-byte upload fails outright because it will never become valid. The download URL is presigned at dispatch and not at creation - a queued job may wait days. ## Job state comes from telemetry, not from the gateway's ok This is the part that is not obvious and was learned the expensive way on 2026-10-15: a real print was accepted, uploaded byte-perfect, reached RUNNING, and then the printer paused itself on a physical broken-filament fault. A job marked `printing` on the gateway's reply would have claimed to be printing for the rest of the day. So `advanceJobsFrom` runs on every telemetry snapshot and the mapping lives in `job_status.ts`, apart from the database so every pair can be tested without a Postgres or a printer. The line that matters: an idle printer does *not* complete a job that is still `uploading`. A printer is idle for the entire transfer - minutes of it, at the 0.2 MiB/s measured over FTPS - so reading that as completion would mark every job successful seconds after dispatch, before the file reached the machine. 15 tests cover the table exhaustively. Because the gateway now acks a start immediately (it must - awaiting the transfer would stop its telemetry for minutes, and telemetry is what this depends on), a failure after the ack has no result message to travel on. Hence the reaper: a job `uploading` for 20 minutes is failed, not requeued, because the printer may actually be printing it and a requeue would put a second job on a working machine. ## Unclaimed printers are visible now `persistTelemetry` still refuses to create printer rows for gateway ids it does not recognise - a typo in a `printers.toml` becoming a permanent phantom printer is worse. But the refusal was silent, which made the worst first-run experience in the platform: gateway connects, reports happily, fleet stays empty, and the only evidence is in stderr. The observation is now kept and served by `GET /farms/:id/unclaimed-printers`, with `gatewayOnline` alongside it because an empty list otherwise cannot distinguish "all claimed" from "no gateway ever connected". Claiming is still `POST /printers`, still owner-only. 77 tests pass.
This commit is contained in:
@@ -6,6 +6,7 @@ import { farms } from "../db/schema";
|
||||
import { farmProjection } from "../db/projections";
|
||||
import { requireFarm, requireOrg, requireUser, visibleFarms } from "../auth";
|
||||
import { PaginationQuery, paginate } from "./pagination";
|
||||
import { isGatewayOnline, listUnclaimedPrinters } from "./ws/edge";
|
||||
|
||||
const IdParams = t.Object({ id: t.String({ format: "uuid" }) });
|
||||
|
||||
@@ -168,4 +169,45 @@ export const farmRoutes = new Elysia({ prefix: "/farms" })
|
||||
summary: "Rotate and return this farm's edge-proxy token (owner only)",
|
||||
},
|
||||
},
|
||||
)
|
||||
/**
|
||||
* Printers this farm's gateway can see that no printer row claims.
|
||||
*
|
||||
* This exists because of the worst first-run experience in the platform.
|
||||
* A gateway reports its fleet by the ids in its own `printers.toml`, and
|
||||
* `persistTelemetry` refuses to create rows for ids it does not recognise -
|
||||
* correctly, since a typo there would otherwise become a permanent phantom
|
||||
* printer that nobody can explain. But the refusal used to be silent, so a
|
||||
* new owner saw a connected gateway, an empty fleet, and no way to find out
|
||||
* why. The answer was in the server's stderr.
|
||||
*
|
||||
* Now it is here. Claim one by creating it with `POST /printers`, passing
|
||||
* the `gatewayPrinterId` from this list; the next telemetry batch fills in
|
||||
* everything else and the entry disappears from here.
|
||||
*
|
||||
* `viewer`, not `owner`: this reports what hardware exists, which is the
|
||||
* same class of fact as the printer list itself. Creating the row still
|
||||
* needs `owner`.
|
||||
*
|
||||
* `gatewayOnline` is in the response because an empty list is ambiguous on
|
||||
* its own - it means either "everything is claimed" or "no gateway has ever
|
||||
* connected", and those need completely different things done about them.
|
||||
*/
|
||||
.get(
|
||||
"/:id/unclaimed-printers",
|
||||
async ({ user, params }) => {
|
||||
await requireFarm(user.id, params.id, "viewer");
|
||||
return {
|
||||
farmId: params.id,
|
||||
gatewayOnline: isGatewayOnline(params.id),
|
||||
unclaimed: listUnclaimedPrinters(params.id),
|
||||
};
|
||||
},
|
||||
{
|
||||
params: IdParams,
|
||||
detail: {
|
||||
tags: ["farms"],
|
||||
summary: "Printers the gateway reports that no printer row claims yet",
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user