octoturge
564d01816b
`POST /jobs` ended at a `queued` row and an upload URL; nothing moved it. This is the join, plus the two things that join turns out to need. ## The scheduler `src/dispatch` ticks every 5s: for each idle, claimed printer on a farm with a connected gateway, take the oldest unassigned queued job and send `start_print`. A timer rather than an event, because a job becomes dispatchable for two unrelated reasons - someone queued it, or a printer became free - and the second has no event this backend owns. It arrives as a telemetry snapshot, seconds after the fact. A scheduler that only ran on job creation would leave a queue stuck forever behind one busy printer. Claiming is an `UPDATE ... WHERE status = 'queued'` that returns the row, so two ticks cannot both win the same job. Selecting and then updating would let a tick and a nudge each believe they owned it and send two starts. It checks the bucket before dispatching. `POST /jobs` hands back a presigned PUT and returns, so the row exists before the file does; dispatching anyway would have the gateway download R2's 403 error document and give it to a printer as a project. A job whose file never arrives just stays queued, and a zero-byte upload fails outright because it will never become valid. The download URL is presigned at dispatch and not at creation - a queued job may wait days. ## Job state comes from telemetry, not from the gateway's ok This is the part that is not obvious and was learned the expensive way on 2026-10-15: a real print was accepted, uploaded byte-perfect, reached RUNNING, and then the printer paused itself on a physical broken-filament fault. A job marked `printing` on the gateway's reply would have claimed to be printing for the rest of the day. So `advanceJobsFrom` runs on every telemetry snapshot and the mapping lives in `job_status.ts`, apart from the database so every pair can be tested without a Postgres or a printer. The line that matters: an idle printer does *not* complete a job that is still `uploading`. A printer is idle for the entire transfer - minutes of it, at the 0.2 MiB/s measured over FTPS - so reading that as completion would mark every job successful seconds after dispatch, before the file reached the machine. 15 tests cover the table exhaustively. Because the gateway now acks a start immediately (it must - awaiting the transfer would stop its telemetry for minutes, and telemetry is what this depends on), a failure after the ack has no result message to travel on. Hence the reaper: a job `uploading` for 20 minutes is failed, not requeued, because the printer may actually be printing it and a requeue would put a second job on a working machine. ## Unclaimed printers are visible now `persistTelemetry` still refuses to create printer rows for gateway ids it does not recognise - a typo in a `printers.toml` becoming a permanent phantom printer is worse. But the refusal was silent, which made the worst first-run experience in the platform: gateway connects, reports happily, fleet stays empty, and the only evidence is in stderr. The observation is now kept and served by `GET /farms/:id/unclaimed-printers`, with `gatewayOnline` alongside it because an empty list otherwise cannot distinguish "all claimed" from "no gateway ever connected". Claiming is still `POST /printers`, still owner-only. 77 tests pass.
73 lines
3.1 KiB
TypeScript
73 lines
3.1 KiB
TypeScript
import { describe, expect, test } from "bun:test";
|
|
import { nextJobStatus } from "../src/dispatch/job_status";
|
|
|
|
/**
|
|
* The job-status mapping, tested exhaustively.
|
|
*
|
|
* This is the logic that decides whether a print happened. It exists because
|
|
* a gateway accepting `start_print` says nothing about the outcome: a real
|
|
* end-to-end run on a P1P (2026-10-15) was accepted, uploaded byte-perfect,
|
|
* reached RUNNING and then the printer paused itself on a physical
|
|
* broken-filament fault. Job state is therefore derived from the telemetry
|
|
* that follows, and getting this table wrong means the API reports prints
|
|
* that never ran - the one failure nobody would catch by looking.
|
|
*/
|
|
describe("nextJobStatus", () => {
|
|
const currents = ["uploading", "printing", "paused"] as const;
|
|
const printerStates = ["idle", "printing", "paused", "error"] as const;
|
|
|
|
// Written out rather than computed: a table derived the same way the
|
|
// implementation derives it agrees with a wrong implementation.
|
|
const expected: Record<string, string | undefined> = {
|
|
"uploading/idle": undefined,
|
|
"uploading/printing": "printing",
|
|
"uploading/paused": "paused",
|
|
"uploading/error": "failed",
|
|
"printing/idle": "completed",
|
|
"printing/printing": "printing",
|
|
"printing/paused": "paused",
|
|
"printing/error": "failed",
|
|
"paused/idle": "completed",
|
|
"paused/printing": "printing",
|
|
"paused/paused": "paused",
|
|
"paused/error": "failed",
|
|
};
|
|
|
|
for (const current of currents) {
|
|
for (const printerState of printerStates) {
|
|
const key = `${current}/${printerState}`;
|
|
test(`${key} -> ${expected[key] ?? "unchanged"}`, () => {
|
|
expect(nextJobStatus(current, printerState)).toBe(expected[key] as never);
|
|
});
|
|
}
|
|
}
|
|
|
|
test("an idle printer does NOT complete a job that was still uploading", () => {
|
|
// The case worth its own test. A printer is idle for the entire transfer
|
|
// - minutes of it, at the 0.2 MiB/s measured over FTPS - so "idle" there
|
|
// means "has not started", not "finished". Reading it as completion
|
|
// would mark every job successful seconds after dispatch, before the
|
|
// file had even reached the machine.
|
|
expect(nextJobStatus("uploading", "idle")).toBeUndefined();
|
|
});
|
|
|
|
test("a printer error fails the job from any state, including mid-upload", () => {
|
|
// The P1P's self-pause arrived while the job was still `uploading` by
|
|
// the cloud's reckoning, because the printer faulted during its
|
|
// pre-print stages. If `uploading` were excluded here, that job would
|
|
// have sat claiming to upload until the reaper got it twenty minutes
|
|
// later.
|
|
for (const current of currents) {
|
|
expect(nextJobStatus(current, "error")).toBe("failed");
|
|
}
|
|
});
|
|
|
|
test("completion is never reported from a state the print had not begun in", () => {
|
|
// Guards the invariant rather than the table: the only route to
|
|
// "completed" is through a state that means the printer had actually
|
|
// been printing.
|
|
const completable = currents.filter((c) => nextJobStatus(c, "idle") === "completed");
|
|
expect(completable).toEqual(["printing", "paused"]);
|
|
});
|
|
});
|