`POST /jobs` ended at a `queued` row and an upload URL; nothing moved it. This is the join, plus the two things that join turns out to need. ## The scheduler `src/dispatch` ticks every 5s: for each idle, claimed printer on a farm with a connected gateway, take the oldest unassigned queued job and send `start_print`. A timer rather than an event, because a job becomes dispatchable for two unrelated reasons - someone queued it, or a printer became free - and the second has no event this backend owns. It arrives as a telemetry snapshot, seconds after the fact. A scheduler that only ran on job creation would leave a queue stuck forever behind one busy printer. Claiming is an `UPDATE ... WHERE status = 'queued'` that returns the row, so two ticks cannot both win the same job. Selecting and then updating would let a tick and a nudge each believe they owned it and send two starts. It checks the bucket before dispatching. `POST /jobs` hands back a presigned PUT and returns, so the row exists before the file does; dispatching anyway would have the gateway download R2's 403 error document and give it to a printer as a project. A job whose file never arrives just stays queued, and a zero-byte upload fails outright because it will never become valid. The download URL is presigned at dispatch and not at creation - a queued job may wait days. ## Job state comes from telemetry, not from the gateway's ok This is the part that is not obvious and was learned the expensive way on 2026-10-15: a real print was accepted, uploaded byte-perfect, reached RUNNING, and then the printer paused itself on a physical broken-filament fault. A job marked `printing` on the gateway's reply would have claimed to be printing for the rest of the day. So `advanceJobsFrom` runs on every telemetry snapshot and the mapping lives in `job_status.ts`, apart from the database so every pair can be tested without a Postgres or a printer. The line that matters: an idle printer does *not* complete a job that is still `uploading`. A printer is idle for the entire transfer - minutes of it, at the 0.2 MiB/s measured over FTPS - so reading that as completion would mark every job successful seconds after dispatch, before the file reached the machine. 15 tests cover the table exhaustively. Because the gateway now acks a start immediately (it must - awaiting the transfer would stop its telemetry for minutes, and telemetry is what this depends on), a failure after the ack has no result message to travel on. Hence the reaper: a job `uploading` for 20 minutes is failed, not requeued, because the printer may actually be printing it and a requeue would put a second job on a working machine. ## Unclaimed printers are visible now `persistTelemetry` still refuses to create printer rows for gateway ids it does not recognise - a typo in a `printers.toml` becoming a permanent phantom printer is worse. But the refusal was silent, which made the worst first-run experience in the platform: gateway connects, reports happily, fleet stays empty, and the only evidence is in stderr. The observation is now kept and served by `GET /farms/:id/unclaimed-printers`, with `gatewayOnline` alongside it because an empty list otherwise cannot distinguish "all claimed" from "no gateway ever connected". Claiming is still `POST /printers`, still owner-only. 77 tests pass.
240 lines
9.5 KiB
TypeScript
240 lines
9.5 KiB
TypeScript
import { describe, expect, it } from "bun:test";
|
|
import { Value } from "@sinclair/typebox/value";
|
|
import { EdgeClientMessageSchema, EdgeServerMessageSchema } from "@continuum/ts-types";
|
|
|
|
import currentCapture from "./fixtures/gateway_frames.json";
|
|
import legacyCapture from "./fixtures/gateway_frames_legacy_nulls.json";
|
|
|
|
/**
|
|
* A contract test, not a unit test: it checks that what continuum-proxy
|
|
* actually puts on the wire is what this backend's validator accepts.
|
|
*
|
|
* The fixtures are real captures from a live gateway talking to a real P1P and
|
|
* H2C, not hand-written examples - which is the whole point. The two ends of
|
|
* /ws/edge/v1 had silently drifted into completely different protocols
|
|
* (different handshake, different telemetry shape, printer ids the gateway
|
|
* could not know), and every hand-written example agreed with whichever side
|
|
* wrote it. Re-capture with scripts/capture-gateway-frames.js when the protocol
|
|
* changes, rather than editing these by hand.
|
|
*/
|
|
|
|
function check(schema: Parameters<typeof Value.Check>[0], value: unknown) {
|
|
const ok = Value.Check(schema, value);
|
|
if (!ok) {
|
|
// Surface *why*, or a failure here is just "false !== true".
|
|
const errors = [...Value.Errors(schema, value)].map(
|
|
(e) => `${e.path || "/"}: ${e.message} (got ${JSON.stringify(e.value)})`,
|
|
);
|
|
throw new Error(`validation failed:\n ${errors.join("\n ")}`);
|
|
}
|
|
}
|
|
|
|
describe("gateway -> backend frames", () => {
|
|
for (const frame of currentCapture.frames) {
|
|
it(`accepts a real '${frame.type}' frame`, () => {
|
|
check(EdgeClientMessageSchema, frame);
|
|
});
|
|
}
|
|
|
|
it("accepts the pre-fix capture, where empty fields were explicit nulls", () => {
|
|
// Liberal reader: the gateway now omits these, but a serde-derived
|
|
// producer emits null by default and one such field must not cost us a
|
|
// whole farm's telemetry batch.
|
|
for (const frame of legacyCapture.frames) {
|
|
check(EdgeClientMessageSchema, frame);
|
|
}
|
|
});
|
|
|
|
it("carries the live telemetry this was captured from", () => {
|
|
const telemetry = currentCapture.frames.find((f) => f.type === "telemetry");
|
|
expect(telemetry).toBeDefined();
|
|
const printers = (telemetry as { printers: Array<{ printer_id: string }> }).printers;
|
|
expect(printers.length).toBeGreaterThan(0);
|
|
// Guards against a fixture that validates because it is empty.
|
|
expect(printers.every((p) => typeof p.printer_id === "string" && p.printer_id.length > 0)).toBe(true);
|
|
});
|
|
|
|
it("accepts a stage, including stage 0", () => {
|
|
// `stage` is Bambu's `stg_cur`: what the printer is doing when it is not
|
|
// laying down plastic. 0 means printing, and a schema or serialiser that
|
|
// treats 0 as absent loses the distinction that makes the field worth
|
|
// having - a printer at 14% and layer 0 is mid-bed-levelling at stage 54
|
|
// and genuinely stuck at stage 0.
|
|
for (const stage of [0, 13, 54]) {
|
|
check(EdgeClientMessageSchema, {
|
|
type: "telemetry",
|
|
printers: [{ printer_id: "p1p", name: "P1P", percent: 14, layer: 0, stage }],
|
|
});
|
|
}
|
|
|
|
// The assertion that makes the loop above mean anything. This schema
|
|
// permits unknown properties, so `{ stage: 54 }` validates whether or
|
|
// not `stage` is declared - verified, it does. Rejecting a *wrongly
|
|
// typed* stage is what proves the field is really in the schema, so this
|
|
// is what fails if someone removes it.
|
|
expect(
|
|
Value.Check(EdgeClientMessageSchema, {
|
|
type: "telemetry",
|
|
printers: [{ printer_id: "p1p", name: "P1P", stage: "54" }],
|
|
}),
|
|
).toBe(false);
|
|
});
|
|
|
|
it("still accepts telemetry with no stage at all", () => {
|
|
// The field is additive: the committed live capture predates it, and a
|
|
// gateway that has not been updated yet simply omits it.
|
|
check(EdgeClientMessageSchema, {
|
|
type: "telemetry",
|
|
printers: [{ printer_id: "p1p", name: "P1P", percent: 0 }],
|
|
});
|
|
});
|
|
|
|
it("rejects a telemetry batch whose printer has no id", () => {
|
|
// printer_id and name are the only required fields; everything else is
|
|
// genuinely unknowable for a printer that has only just connected.
|
|
check(EdgeClientMessageSchema, { type: "telemetry", printers: [{ printer_id: "p1p", name: "P1P" }] });
|
|
expect(Value.Check(EdgeClientMessageSchema, { type: "telemetry", printers: [{ name: "P1P" }] })).toBe(false);
|
|
});
|
|
|
|
it("rejects an unknown message type", () => {
|
|
// The old protocol's handshake, which no gateway ever sent.
|
|
expect(Value.Check(EdgeClientMessageSchema, { type: "auth", token: "x" })).toBe(false);
|
|
});
|
|
});
|
|
|
|
describe("backend -> gateway frames", () => {
|
|
// Exactly what src/routes/ws/edge.ts sends. If one of these stops
|
|
// validating, the gateway's serde enum will have stopped accepting it too -
|
|
// see the mirror of this test in continuum-proxy/tests/uplink_protocol.rs.
|
|
const outbound = [
|
|
{ type: "hello_ack", session_id: "3f2b1c5e-0000-4000-8000-000000000000" },
|
|
{ type: "hello_error", message: "invalid edge token" },
|
|
{ type: "heartbeat_ack", seq: 7 },
|
|
{
|
|
type: "command",
|
|
command_id: "9a7d2f10-0000-4000-8000-000000000000",
|
|
printer_id: "p1p",
|
|
command: { action: "set_fan_speed", percent: 40 },
|
|
},
|
|
{
|
|
type: "command",
|
|
command_id: "9a7d2f11-0000-4000-8000-000000000000",
|
|
printer_id: "p1p",
|
|
command: { action: "send_gcode_line", line: "M106 S0" },
|
|
},
|
|
{
|
|
type: "command",
|
|
command_id: "9a7d2f12-0000-4000-8000-000000000000",
|
|
printer_id: "p1p",
|
|
command: { action: "control_print", control: "pause" },
|
|
},
|
|
{
|
|
type: "command",
|
|
command_id: "9a7d2f13-0000-4000-8000-000000000000",
|
|
printer_id: "p1p",
|
|
command: { action: "control_print", control: "resume" },
|
|
},
|
|
{
|
|
type: "command",
|
|
command_id: "9a7d2f14-0000-4000-8000-000000000000",
|
|
printer_id: "p1p",
|
|
command: { action: "control_print", control: "stop" },
|
|
},
|
|
{
|
|
type: "command",
|
|
command_id: "9a7d2f15-0000-4000-8000-000000000000",
|
|
printer_id: "p1p",
|
|
command: {
|
|
action: "start_print",
|
|
job_id: "7c9e6679-0000-4000-8000-000000000000",
|
|
file_name: "cube.gcode.3mf",
|
|
// Exactly what src/dispatch sends: a presigned GET, minted at
|
|
// dispatch time rather than at job creation.
|
|
url: "https://example.r2.cloudflarestorage.com/farms/f/jobs/j/cube.gcode.3mf?X-Amz-Signature=x",
|
|
},
|
|
},
|
|
];
|
|
|
|
for (const frame of outbound) {
|
|
it(`emits a valid '${frame.type}' (${(frame as { command?: { action: string } }).command?.action ?? "no command"})`, () => {
|
|
check(EdgeServerMessageSchema, frame);
|
|
});
|
|
}
|
|
|
|
it("rejects a print control verb the gateway does not implement", () => {
|
|
// "cancel" is the trap: it is what Moonraker calls this, and the Klipper
|
|
// implementation behind `control_print` really does send a cancel - but
|
|
// the wire word is "stop". A verb that only one vendor recognises must
|
|
// not validate here, or it reaches a gateway that cannot map it.
|
|
for (const control of ["cancel", "abort", "Pause", ""]) {
|
|
expect(
|
|
Value.Check(EdgeServerMessageSchema, {
|
|
type: "command",
|
|
command_id: "x",
|
|
printer_id: "p1p",
|
|
command: { action: "control_print", control },
|
|
}),
|
|
).toBe(false);
|
|
}
|
|
});
|
|
|
|
it("rejects a start_print missing any of its three required fields", () => {
|
|
// The variant exists now - the job scheduler in src/dispatch sends it.
|
|
// What this guards is that none of its fields is optional. Each one is
|
|
// load-bearing in a different way: without `url` the gateway has nothing
|
|
// to fetch, without `file_name` it has nowhere to put it, and without
|
|
// `job_id` nothing can line a gateway's logs up against a job. An
|
|
// optional field here would fail at the gateway, minutes later, on a
|
|
// machine nobody is watching.
|
|
const complete = {
|
|
action: "start_print",
|
|
job_id: "7c9e6679-0000-4000-8000-000000000000",
|
|
file_name: "plate_1.3mf",
|
|
url: "https://example.test/signed",
|
|
};
|
|
check(EdgeServerMessageSchema, {
|
|
type: "command",
|
|
command_id: "x",
|
|
printer_id: "p1p",
|
|
command: complete,
|
|
});
|
|
|
|
for (const omit of ["job_id", "file_name", "url"] as const) {
|
|
const partial: Record<string, unknown> = { ...complete };
|
|
delete partial[omit];
|
|
expect(
|
|
Value.Check(EdgeServerMessageSchema, {
|
|
type: "command",
|
|
command_id: "x",
|
|
printer_id: "p1p",
|
|
command: partial,
|
|
}),
|
|
).toBe(false);
|
|
}
|
|
});
|
|
|
|
it("will not carry a filament or AMS mapping on a start_print", () => {
|
|
// Deliberately absent from the contract, and the reason is expensive:
|
|
// hardcoding the filament choice instead of reading it from the project
|
|
// archive made a real P1P heat to 250C and pause itself at 1%. The
|
|
// gateway derives it from the `.3mf`'s own slice_info.config, which is
|
|
// the only thing that knows. A field here would invite a caller to guess
|
|
// again - and because this schema is a closed union, an extra property
|
|
// makes the whole command fail to validate rather than being ignored.
|
|
expect(
|
|
Value.Check(EdgeServerMessageSchema, {
|
|
type: "command",
|
|
command_id: "x",
|
|
printer_id: "p1p",
|
|
command: {
|
|
action: "start_print",
|
|
job_id: "7c9e6679-0000-4000-8000-000000000000",
|
|
file_name: "plate_1.3mf",
|
|
url: "https://example.test/signed",
|
|
ams_mapping: [3, -1, -1, -1, -1],
|
|
},
|
|
}),
|
|
).toBe(false);
|
|
});
|
|
});
|