Continuum/continuum-backend
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Verify the edge token at the upgrade, not in the first frame

Browse Source 
`/ws/edge/v1` took its credential from a `token` field on the `hello`
message. It now takes it from `Authorization: Bearer` on the upgrade
request, verified before the socket exists, so an unauthenticated peer
gets an HTTP 401 and no WebSocket: no entry in `connections`, no frame
ever parsed, no state of any kind allocated for it.

The old arrangement worked, and every message type refused to act
without a verified farm. But "it cannot do anything" is a weaker
guarantee than "it does not exist" - an anonymous peer still got a live
socket and had its JSON run through `EdgeClientMessageSchema` and the
message switch before proving anything, and it took a timeout to stop
anyone holding sockets open for free. A `?token=` query parameter was
never on the table: a bearer credential in a URL lands in access logs,
proxy logs and error breadcrumbs, which is the thing the proxy's
hand-written `Debug` for `PrinterCommand` already exists to prevent.

The decision lives in `src/auth/edge_upgrade.ts`, a leaf module with no
imports but the error types - the same reason as `dispatch/job_status.ts`
and `stream.ts`. This is the authentication boundary for the entire
gateway protocol, so it should be the last thing only reachable through
a running database. It checks `scope` and not just the signature (the
same secret could be made to sign something else one day), rejects a
token whose farm claim is absent or not a string (that would otherwise
produce a connection whose `farmId` is `undefined`, which reads as
"every farm" to a lookup), and answers every failure with one message -
which part of a forged token to fix next is not something to tell its
holder.

`conn.farmId` is therefore set from the verified token in `open`, and
the `hello` handler no longer verifies anything. It still checks
`message.farm_id` against the token's farm, so a gateway naming a farm
its token was not minted for is refused rather than silently working
against the wrong one. The handshake timer stays but changed meaning: it
is no longer an auth timeout, just a bound on an authenticated socket
that never identifies its gateway.

Verified, not assumed. That a throwing `resolve` fails the upgrade is a
framework behaviour the whole design rests on, so `tests/
edge_upgrade.test.ts` stands up a real Elysia app with a real JWT plugin
and asserts a socket never opens without a valid edge token, and that a
resolved value reaches `ws.data`. The real `continuum-proxy` was also run
against this composition: valid token gives HTTP 101 with the farm taken
from the token, forged token gives 401 at the handshake.

The committed captures predate this and still contain `token:
"dev-token"`. That validates - the schema is a liberal reader - and is
harmless, because the field is now ignored and such a gateway is refused
at the upgrade for having no header. A test says so.

95 tests (was 85), `bunx tsc --noEmit` clean.
This commit is contained in:
octoturge
2026-10-17 13:59:56 +00:00
parent 7a0e8ba6ef
commit 57af710b1a
4 changed files with 303 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/auth/edge_upgrade.ts
+61
View File
@@ -0,0 +1,61 @@
import { UnauthorizedError, extractBearerToken } from "@continuum/ts-core";
/**
* Edge-token checking for the `/ws/edge/v1` upgrade, with no imports beyond
* the error types - the same reason `dispatch/job_status.ts` and `stream.ts`
* are leaf modules: `../db` opens a connection at import time, and this is
* the authentication boundary for the whole gateway protocol, so it is the
* last thing that should only be reachable through a running database.
*/
/**
* Whatever verifies an edge JWT. Narrowed to what this needs rather than
* taking the `@elysiajs/jwt` decorator's type, so a test can pass a stub and
* exercise the real decision logic.
*/
export type EdgeTokenVerifier = (
token: string,
) => Promise<Record<string, unknown> | false | null | undefined>;
/**
* The farm a verified edge token belongs to, or an `UnauthorizedError`.
*
* Every failure gets one message. Which part of a forged token to fix next -
* signature, scope, expiry - is not something to tell whoever is holding
* one; the reason is logged instead. Same reasoning as `requireUser`.
*
* The farm is returned rather than the claims: it is the only thing the
* socket needs, and handing back the whole payload invites a later caller to
* start trusting some other claim without deciding to.
*/
export async function farmFromEdgeUpgrade(
authorization: string | null | undefined,
verify: EdgeTokenVerifier,
): Promise<string> {
let token: string;
try {
token = extractBearerToken(authorization);
} catch {
throw new UnauthorizedError("Authorization header must be 'Bearer <edge token>'");
}
const payload = await verify(token);
// `scope` is checked, not just the signature: the same secret could be
// made to sign something else one day, and a token minted for another
// purpose must not open a farm's gateway socket.
if (!payload || payload.scope !== "edge") {
console.warn("[edge] upgrade rejected: token missing or not scoped to edge");
throw new UnauthorizedError("Invalid or expired edge token");
}
// A token whose farm claim is absent or not a string would otherwise
// produce a connection whose `farmId` is `undefined`, which reads as
// "every farm" to anything doing a lookup.
if (typeof payload.farmId !== "string" || payload.farmId.length === 0) {
console.warn("[edge] upgrade rejected: edge token carries no farm");
throw new UnauthorizedError("Invalid or expired edge token");
}
return payload.farmId;
}
src/routes/ws/edge.ts
+72 -31
View File
@@ -12,6 +12,7 @@ import { printers } from "../../db/schema";
import { advanceJobsFrom } from "../../dispatch";
import { enqueueFrameIngestionBatch } from "../../queue";
import { edgeJwtSecret } from "../../auth/edge_secret";
import { farmFromEdgeUpgrade } from "../../auth/edge_upgrade";
/**
* Edge-gateway uplink. A continuum-proxy gateway connects here with a
@@ -19,6 +20,37 @@ import { edgeJwtSecret } from "../../auth/edge_secret";
* for its whole printer fleet, and receives commands for individual printers
* on the same socket.
*
* ## Where the token goes, and why here
*
* `Authorization: Bearer <edge jwt>` on the upgrade request, verified before
* the socket exists. An unauthenticated peer therefore gets an HTTP 401 and
* no WebSocket at all: no entry in `connections`, no frame ever parsed, no
* state of any kind allocated for it.
*
* The two alternatives were both weaker, and one of them is what this used
* to do:
*
* - **In the first message** (a `token` field on `hello`, which is what this
* route did until now). It worked, but it meant an unauthenticated peer
* got a live socket, an entry in `connections`, and its JSON run through
* `EdgeClientMessageSchema` and this switch statement before proving
* anything. Every message type refused to act without a verified farm, so
* nothing could be *done* with it - but "it cannot do anything" is a
* weaker guarantee than "it does not exist", and it needed a timeout to
* stop anyone holding sockets open for free.
* - **A `?token=` query parameter.** A bearer credential in a URL ends up in
* access logs, proxy logs and error breadcrumbs. This codebase already
* goes to some trouble to avoid exactly that - `PrinterCommand` in the
* proxy has a hand-written `Debug` so a presigned URL prints as
* `<redacted>`. `Authorization` is the header every log tool knows to
* redact; a query string is not.
*
* A browser cannot set headers on a WebSocket, so a future user-facing
* socket cannot copy this exactly. Its equivalent is the subprotocol field
* (`new WebSocket(url, ["bearer", token])`), which is also verified at the
* upgrade - the same property, through the one handshake field a browser
* does control.
*
* The wire format is `schemas/continuum/v1/edge/` in continuum-schemas,
* validated here through its TypeBox mirror in `@continuum/ts-types`. It is
* snake_case, unlike this backend's HTTP routes: the other end is
@@ -49,16 +81,16 @@ interface EdgeConn {
*/
streamBaseUrl?: string;
/**
* Closes this socket if it never authenticates. Cleared by a successful
* `hello` and on close.
* Closes this socket if it never identifies itself. Cleared by a
* successful `hello` and on close.
*
* The cost of carrying the token in the first message rather than the
* handshake is exactly this: there is a window in which an open socket
* belongs to nobody. Every other message type already refuses to act
* without a `farmId`, so such a socket can do nothing - but it does sit in
* `connections` forever, and that map is keyed by whoever can reach the
* port. A silent connection is not a slow one: a real gateway sends
* `hello` as its first frame.
* No longer an authentication timeout - the token is verified at the
* upgrade, so every socket here already belongs to a farm. What it bounds
* now is a protocol question: an authenticated socket that never sends
* `hello` has no `gateway_id`, so nothing can be logged about it usefully
* and it is holding a connection it is not using. Kept because a real
* gateway sends `hello` as its first frame, so silence means something is
* wrong rather than slow.
*/
handshakeTimer?: ReturnType<typeof setTimeout>;
send: (message: EdgeServerMessage) => void;
@@ -150,10 +182,10 @@ export function listUnclaimedPrinters(farmId: string): UnclaimedPrinter[] {
const COMMAND_TIMEOUT_MS = 10_000;
/**
* How long a socket may stay open without authenticating. Generous compared
* with the round trip it covers - the gateway sends `hello` immediately after
* the socket opens - because the only thing a tight bound would buy is
* dropping gateways on slow links.
* How long an authenticated socket may stay open without sending `hello`.
* Generous compared with the round trip it covers - a gateway sends `hello`
* immediately after the socket opens - because the only thing a tight bound
* would buy is dropping gateways on slow links.
*/
const HANDSHAKE_TIMEOUT_MS = 10_000;
@@ -333,14 +365,30 @@ export function getStreamBaseUrl(farmId: string): string | undefined {
export const edgeWsRoute = new Elysia()
.use(jwt({ name: "edgeJwt", secret: edgeJwtSecret }))
/**
* Verifies the edge JWT on the upgrade request, before any socket exists.
*
* Throwing here fails the upgrade with an HTTP status, so a peer without a
* valid token never reaches `open` - which is the whole point of
* authenticating at the handshake rather than in the first frame. Modelled
* on `requireUser`'s scoped resolve, for the same reason: one verification
* per connection, in one place, ahead of every handler.
*/
.resolve({ as: "scoped" }, async ({ headers, edgeJwt }): Promise<{ edgeFarmId: string }> => ({
edgeFarmId: await farmFromEdgeUpgrade(headers.authorization, (token) => edgeJwt.verify(token)),
}))
.ws("/ws/edge/v1", {
body: EdgeClientMessageSchema,
open(ws) {
// No farmId yet - set by `hello`, and every other message type refuses
// to do anything until it is.
// The farm comes from the verified token, so it is known before the
// first frame. `gatewayId` still waits for `hello`: which gateway of a
// farm's this is, and where it serves camera streams, are things only
// the gateway can say.
const conn: EdgeConn = {
farmId: "",
farmId: ws.data.edgeFarmId,
gatewayId: "",
sessionId: crypto.randomUUID(),
lastSeenAt: Date.now(),
@@ -350,7 +398,7 @@ export const edgeWsRoute = new Elysia()
};
conn.handshakeTimer = setTimeout(() => {
if (conn.farmId) return;
if (conn.gatewayId) return;
console.warn(`[edge] closing socket ${ws.id}: no hello within ${HANDSHAKE_TIMEOUT_MS}ms`);
ws.close();
}, HANDSHAKE_TIMEOUT_MS);
@@ -366,31 +414,24 @@ export const edgeWsRoute = new Elysia()
switch (message.type) {
case "hello": {
const payload = await ws.data.edgeJwt.verify(message.token);
if (!payload || payload.scope !== "edge") {
ws.send({ type: "hello_error", message: "invalid edge token" });
ws.close();
return;
}
// The token decides the farm, never the message body: a gateway
// claiming a farm_id its token was not minted for is exactly what
// this check is here to stop.
const farmId = payload.farmId as string;
// Already authenticated at the upgrade, so there is no token to
// check here. The farm still comes from that token and never from
// the message body: a gateway claiming a farm_id its token was not
// minted for is exactly what the next check is here to stop.
const farmId = conn.farmId;
if (message.farm_id && message.farm_id !== farmId) {
ws.send({ type: "hello_error", message: "farm_id does not match the token's farm" });
ws.close();
return;
}
// Authenticated, so it is no longer a socket belonging to nobody.
// Cleared only here, after the token verified and the farm matched
// - a rejected hello leaves the timer running, so a client that
// retries a bad token on one socket still gets dropped.
// Identified, so the timer has nothing left to catch. Cleared
// only after the farm matched - a rejected hello leaves it
// running, so a client that keeps sending mismatched farm ids on
// one socket still gets dropped.
clearTimeout(conn.handshakeTimer);
conn.handshakeTimer = undefined;
conn.farmId = farmId;
conn.gatewayId = message.gateway_id;
// `?? undefined`: absent and explicit null both mean "no streams".
conn.streamBaseUrl = message.stream_base_url ?? undefined;