`POST /jobs` ended at a `queued` row and an upload URL; nothing moved it. This is the join, plus the two things that join turns out to need. ## The scheduler `src/dispatch` ticks every 5s: for each idle, claimed printer on a farm with a connected gateway, take the oldest unassigned queued job and send `start_print`. A timer rather than an event, because a job becomes dispatchable for two unrelated reasons - someone queued it, or a printer became free - and the second has no event this backend owns. It arrives as a telemetry snapshot, seconds after the fact. A scheduler that only ran on job creation would leave a queue stuck forever behind one busy printer. Claiming is an `UPDATE ... WHERE status = 'queued'` that returns the row, so two ticks cannot both win the same job. Selecting and then updating would let a tick and a nudge each believe they owned it and send two starts. It checks the bucket before dispatching. `POST /jobs` hands back a presigned PUT and returns, so the row exists before the file does; dispatching anyway would have the gateway download R2's 403 error document and give it to a printer as a project. A job whose file never arrives just stays queued, and a zero-byte upload fails outright because it will never become valid. The download URL is presigned at dispatch and not at creation - a queued job may wait days. ## Job state comes from telemetry, not from the gateway's ok This is the part that is not obvious and was learned the expensive way on 2026-10-15: a real print was accepted, uploaded byte-perfect, reached RUNNING, and then the printer paused itself on a physical broken-filament fault. A job marked `printing` on the gateway's reply would have claimed to be printing for the rest of the day. So `advanceJobsFrom` runs on every telemetry snapshot and the mapping lives in `job_status.ts`, apart from the database so every pair can be tested without a Postgres or a printer. The line that matters: an idle printer does *not* complete a job that is still `uploading`. A printer is idle for the entire transfer - minutes of it, at the 0.2 MiB/s measured over FTPS - so reading that as completion would mark every job successful seconds after dispatch, before the file reached the machine. 15 tests cover the table exhaustively. Because the gateway now acks a start immediately (it must - awaiting the transfer would stop its telemetry for minutes, and telemetry is what this depends on), a failure after the ack has no result message to travel on. Hence the reaper: a job `uploading` for 20 minutes is failed, not requeued, because the printer may actually be printing it and a requeue would put a second job on a working machine. ## Unclaimed printers are visible now `persistTelemetry` still refuses to create printer rows for gateway ids it does not recognise - a typo in a `printers.toml` becoming a permanent phantom printer is worse. But the refusal was silent, which made the worst first-run experience in the platform: gateway connects, reports happily, fleet stays empty, and the only evidence is in stderr. The observation is now kept and served by `GET /farms/:id/unclaimed-printers`, with `gatewayOnline` alongside it because an empty list otherwise cannot distinguish "all claimed" from "no gateway ever connected". Claiming is still `POST /printers`, still owner-only. 77 tests pass.
214 lines
7.9 KiB
TypeScript
214 lines
7.9 KiB
TypeScript
import { Elysia, t } from "elysia";
|
|
import { and, desc, eq } from "drizzle-orm";
|
|
import { NotFoundError } from "@continuum/ts-core";
|
|
import { db } from "../db";
|
|
import { farms } from "../db/schema";
|
|
import { farmProjection } from "../db/projections";
|
|
import { requireFarm, requireOrg, requireUser, visibleFarms } from "../auth";
|
|
import { PaginationQuery, paginate } from "./pagination";
|
|
import { isGatewayOnline, listUnclaimedPrinters } from "./ws/edge";
|
|
|
|
const IdParams = t.Object({ id: t.String({ format: "uuid" }) });
|
|
|
|
export const farmRoutes = new Elysia({ prefix: "/farms" })
|
|
.use(requireUser)
|
|
/**
|
|
* Farms in the caller's organisations. The scoping is in the `where`, not
|
|
* applied to the result afterwards: a filter that runs after the query is
|
|
* one `return` away from being skipped, and it also makes the database read
|
|
* rows it is not allowed to show.
|
|
*
|
|
* A projection rather than `select *`, because the table holds
|
|
* `edgeProxyToken`, which trades for an edge JWT - see src/db/projections.ts.
|
|
*/
|
|
.get(
|
|
"/",
|
|
async ({ user, query }) => {
|
|
const scope = query.orgId
|
|
? and(visibleFarms(user.id), eq(farms.orgId, query.orgId))
|
|
: visibleFarms(user.id);
|
|
return db.query.farms.findMany({
|
|
columns: farmProjection.columns,
|
|
where: scope,
|
|
orderBy: [desc(farms.createdAt)],
|
|
...paginate(query),
|
|
});
|
|
},
|
|
{
|
|
query: t.Object({
|
|
orgId: t.Optional(t.String({ format: "uuid" })),
|
|
...PaginationQuery,
|
|
}),
|
|
detail: { tags: ["farms"], summary: "Farms in the caller's organisations" },
|
|
},
|
|
)
|
|
.get(
|
|
"/:id",
|
|
async ({ user, params }) => {
|
|
const { role } = await requireFarm(user.id, params.id, "viewer");
|
|
const farm = await db.query.farms.findFirst({
|
|
columns: farmProjection.columns,
|
|
where: eq(farms.id, params.id),
|
|
});
|
|
if (!farm) throw new NotFoundError(`Farm ${params.id} not found`);
|
|
return { ...farm, role };
|
|
},
|
|
{ params: IdParams, detail: { tags: ["farms"] } },
|
|
)
|
|
.post(
|
|
"/",
|
|
async ({ user, body, set }) => {
|
|
// `orgId` comes from the body and is then authorized, which is the
|
|
// right way round: the caller says where the farm should go, and the
|
|
// check is whether they own that organisation. What must never come
|
|
// from the body is *who the caller is* - that is `user.id`, from a
|
|
// verified token, and it is why there is no `ownerId` field here any
|
|
// more.
|
|
await requireOrg(user.id, body.orgId, "owner", "creating a farm");
|
|
|
|
// The generated token is deliberately not returned. A farm is created
|
|
// far more often than a gateway is provisioned, and a credential that
|
|
// comes back from a routine create ends up in logs, browser history and
|
|
// screenshots. Fetch it when a gateway actually needs it, below.
|
|
const [farm] = await db
|
|
.insert(farms)
|
|
.values({
|
|
name: body.name,
|
|
orgId: body.orgId,
|
|
edgeProxyToken: crypto.randomUUID(),
|
|
location: body.location,
|
|
})
|
|
.returning(farmProjection.returning);
|
|
set.status = 201;
|
|
return farm;
|
|
},
|
|
{
|
|
body: t.Object({
|
|
name: t.String({ minLength: 1, maxLength: 200 }),
|
|
orgId: t.String({ format: "uuid" }),
|
|
location: t.Optional(t.String({ maxLength: 500 })),
|
|
}),
|
|
detail: { tags: ["farms"] },
|
|
},
|
|
)
|
|
.patch(
|
|
"/:id",
|
|
async ({ user, params, body }) => {
|
|
await requireFarm(user.id, params.id, "owner", "editing a farm");
|
|
// `orgId` is not patchable here on purpose: moving a farm between
|
|
// organisations changes who can see its print history and would need
|
|
// the `owner` role in *both*, so it wants its own endpoint rather than
|
|
// a field on a general update.
|
|
const [farm] = await db
|
|
.update(farms)
|
|
.set({ ...body, updatedAt: new Date() })
|
|
.where(eq(farms.id, params.id))
|
|
.returning(farmProjection.returning);
|
|
if (!farm) throw new NotFoundError(`Farm ${params.id} not found`);
|
|
return farm;
|
|
},
|
|
{
|
|
params: IdParams,
|
|
body: t.Object({
|
|
name: t.Optional(t.String({ minLength: 1, maxLength: 200 })),
|
|
location: t.Optional(t.String({ maxLength: 500 })),
|
|
settings: t.Optional(t.Record(t.String(), t.Unknown())),
|
|
}),
|
|
detail: { tags: ["farms"] },
|
|
},
|
|
)
|
|
.delete(
|
|
"/:id",
|
|
async ({ user, params }) => {
|
|
await requireFarm(user.id, params.id, "owner", "deleting a farm");
|
|
// Cascades to the farm's printers and print jobs, per the schema's
|
|
// foreign keys. Worth knowing before calling it.
|
|
await db.delete(farms).where(eq(farms.id, params.id));
|
|
return { deleted: params.id };
|
|
},
|
|
{ params: IdParams, detail: { tags: ["farms"] } },
|
|
)
|
|
/**
|
|
* Issues a *new* `edgeProxyToken` for this farm and returns it once.
|
|
*
|
|
* The only route that ever returns that credential, and it rotates on every
|
|
* call rather than revealing the stored one: a reveal endpoint returns the
|
|
* same secret to everyone who asks and leaves no trace of having been
|
|
* called, while a rotate endpoint makes the old value useless, so a
|
|
* gateway's token leaking has a bounded blast radius and reprovisioning is
|
|
* the fix.
|
|
*
|
|
* The cost is that rotating locks out the gateway currently using the old
|
|
* token until it is reconfigured, which is the right trade for a credential
|
|
* that drives physical machines - but it does mean this is not a "show me
|
|
* the token" button.
|
|
*/
|
|
.post(
|
|
"/:id/edge-proxy-token",
|
|
async ({ user, params }) => {
|
|
await requireFarm(user.id, params.id, "owner", "rotating a farm's gateway credential");
|
|
|
|
const token = crypto.randomUUID();
|
|
const [farm] = await db
|
|
.update(farms)
|
|
.set({ edgeProxyToken: token, updatedAt: new Date() })
|
|
.where(eq(farms.id, params.id))
|
|
.returning({ id: farms.id });
|
|
if (!farm) throw new NotFoundError(`Farm ${params.id} not found`);
|
|
|
|
return {
|
|
farmId: farm.id,
|
|
edgeProxyToken: token,
|
|
note: "Any gateway still using the previous token must be reconfigured.",
|
|
};
|
|
},
|
|
{
|
|
params: IdParams,
|
|
detail: {
|
|
tags: ["farms"],
|
|
summary: "Rotate and return this farm's edge-proxy token (owner only)",
|
|
},
|
|
},
|
|
)
|
|
/**
|
|
* Printers this farm's gateway can see that no printer row claims.
|
|
*
|
|
* This exists because of the worst first-run experience in the platform.
|
|
* A gateway reports its fleet by the ids in its own `printers.toml`, and
|
|
* `persistTelemetry` refuses to create rows for ids it does not recognise -
|
|
* correctly, since a typo there would otherwise become a permanent phantom
|
|
* printer that nobody can explain. But the refusal used to be silent, so a
|
|
* new owner saw a connected gateway, an empty fleet, and no way to find out
|
|
* why. The answer was in the server's stderr.
|
|
*
|
|
* Now it is here. Claim one by creating it with `POST /printers`, passing
|
|
* the `gatewayPrinterId` from this list; the next telemetry batch fills in
|
|
* everything else and the entry disappears from here.
|
|
*
|
|
* `viewer`, not `owner`: this reports what hardware exists, which is the
|
|
* same class of fact as the printer list itself. Creating the row still
|
|
* needs `owner`.
|
|
*
|
|
* `gatewayOnline` is in the response because an empty list is ambiguous on
|
|
* its own - it means either "everything is claimed" or "no gateway has ever
|
|
* connected", and those need completely different things done about them.
|
|
*/
|
|
.get(
|
|
"/:id/unclaimed-printers",
|
|
async ({ user, params }) => {
|
|
await requireFarm(user.id, params.id, "viewer");
|
|
return {
|
|
farmId: params.id,
|
|
gatewayOnline: isGatewayOnline(params.id),
|
|
unclaimed: listUnclaimedPrinters(params.id),
|
|
};
|
|
},
|
|
{
|
|
params: IdParams,
|
|
detail: {
|
|
tags: ["farms"],
|
|
summary: "Printers the gateway reports that no printer row claims yet",
|
|
},
|
|
},
|
|
);
|