Show the picture in the real web console, in a real browser

Every earlier camera proof stood in for the browser with a werift peer.
The question that decides whether the feature works is whether a person
looking at the fleet page sees a picture, and the web console had never
been opened.

BROWSER=1 on scripts/e2e-camera.sh now starts the real Nuxt app and
drives it in real Chrome (puppeteer-core): sign in, open the fleet page,
wait for the camera player, and check the result. Measured against the
H2C: the app's own Supabase client signs in, the header reports Live
(the telemetry socket authenticated with the real token), the player
reports connected, Chrome decodes 1680x1080 frames, playback time
advances, the frame has picture content (mean luma 88, stdev 62.8), and
the browser's own ICE stats show the selected candidate is a TURN relay.
No uncaught page errors.

The login page only offers a magic link by email, which a script cannot
follow, so the driver signs in with the app's $supabase.client and a
password just set on a throwaway user. That means AUTH=supabase: the
backend verifies against the real project's JWKS rather than the local
dev issuer, with e2e-supabase-user.ts creating or reusing
continuum-e2e@example.invalid (a reserved .invalid address) and
resetting its password each run. The JSON it prints carries a password
and a live token, so the shell captures it and never echoes it.

The frame is judged by pixel statistics and no screenshot is kept: it is
the inside of someone's workshop, and "is there a picture" does not need
one stored. The relay is read from getStats() through a wrapped
RTCPeerConnection rather than inferred from configuration. A browser run
with fewer than its nine results fails, so a driver that dies quietly
cannot read as a pass.

puppeteer-core is a devDependency for these scripts alone. Its 22
resolved packages are all Apache-2.0, BSD-3-Clause, ISC or MIT.

scripts/e2e-camera-browser.ts

@@ -0,0 +1,186 @@
+/**
+ * Drives the real Continuum web console in a real Chrome, for
+ * scripts/e2e-camera.sh (BROWSER=1).
+ *
+ * Everything before this proved the camera path with a stand-in for the
+ * browser. This is the browser: the actual Nuxt app, the actual
+ * `WebRTCStreamView`, a real RTCPeerConnection pinned to relay-only, and Chrome's
+ * own H.264 decoder. The question it answers is whether a person looking at
+ * the fleet page would see a picture.
+ *
+ * Signing in: the login page only offers a magic link by email, which a
+ * script cannot follow, so this signs in with the app's own Supabase client
+ * (`$supabase.client`) using a password the caller just set on a throwaway
+ * user. The app then behaves exactly as it does for a real session: cookie,
+ * redirect, token on every API call, the telemetry socket.
+ *
+ * What it reads from the browser rather than trusting the UI: the page wraps
+ * RTCPeerConnection so the selected ICE candidate pair can be inspected, and
+ * the decoded frame is sampled on a canvas. It deliberately saves no
+ * screenshot - the frame is the inside of someone's workshop, and pixel
+ * statistics answer "is there a picture" without keeping one.
+ *
+ * Results are `ok<TAB>label` / `FAILED<TAB>label`. The password and token
+ * never reach a log line.
+ *
+ * Environment: APP_URL, SUPABASE_E2E_EMAIL, SUPABASE_E2E_PASSWORD, CHROME_BIN.
+ */
+import puppeteer from "puppeteer-core";
+
+const appUrl = process.env.APP_URL!;
+const email = process.env.SUPABASE_E2E_EMAIL!;
+const password = process.env.SUPABASE_E2E_PASSWORD!;
+const chromeBin = process.env.CHROME_BIN ?? "/usr/bin/google-chrome";
+
+let failed = false;
+const say = (good: boolean, label: string, detail = "") => {
+  if (!good) failed = true;
+  console.log(`${good ? "ok" : "FAILED"}\t${label}${detail ? ` (${detail})` : ""}`);
+};
+
+const browser = await puppeteer.launch({
+  executablePath: chromeBin,
+  headless: true,
+  // No sandbox because this container cannot create the namespaces Chrome's
+  // sandbox needs (see continuum-local-env-limits); the page it loads is our
+  // own dev server.
+  args: ["--no-sandbox", "--disable-dev-shm-usage", "--autoplay-policy=no-user-gesture-required"],
+});
+
+try {
+  const page = await browser.newPage();
+
+  const pageErrors: string[] = [];
+  page.on("pageerror", (err) => pageErrors.push(String((err as Error).message ?? err).slice(0, 160)));
+
+  // Keeps every RTCPeerConnection the app creates, so its stats can be read.
+  await page.evaluateOnNewDocument(() => {
+    const Native = window.RTCPeerConnection;
+    (window as any).__pcs = [];
+    (window as any).RTCPeerConnection = function (this: unknown, ...args: any[]) {
+      const pc = new Native(...(args as [RTCConfiguration?]));
+      (window as any).__pcs.push(pc);
+      return pc;
+    } as unknown as typeof RTCPeerConnection;
+    (window as any).RTCPeerConnection.prototype = Native.prototype;
+  });
+
+  // The first load compiles the app, so it is slow.
+  await page.goto(`${appUrl}/login`, { waitUntil: "domcontentloaded", timeout: 120_000 });
+  await page.waitForFunction(
+    () => typeof (window as any).useNuxtApp === "function" && Boolean((window as any).useNuxtApp().$supabase),
+    { timeout: 120_000 },
+  );
+
+  const signInError = await page.evaluate(
+    async (e: string, p: string) => {
+      const { client } = (window as any).useNuxtApp().$supabase;
+      const { error } = await client.auth.signInWithPassword({ email: e, password: p });
+      return error ? String(error.message) : null;
+    },
+    email,
+    password,
+  );
+  say(signInError === null, "the app's own Supabase client signs the user in", signInError ?? "");
+  if (signInError) process.exit(1);
+
+  await page.goto(`${appUrl}/`, { waitUntil: "domcontentloaded", timeout: 60_000 });
+  const stayed = await page
+    .waitForSelector(".fleet", { timeout: 60_000 })
+    .then(() => true)
+    .catch(() => false);
+  say(stayed && !page.url().includes("/login"), "the fleet page loads and does not bounce back to /login", page.url());
+
+  const live = await page
+    .waitForFunction(() => document.querySelector(".app-shell__status.is-online") !== null, { timeout: 30_000 })
+    .then(() => true)
+    .catch(() => false);
+  say(live, "the header reports Live: the telemetry socket authenticated with the real token");
+
+  const connected = await page
+    .waitForFunction(() => document.querySelector(".webrtc-view__status.is-connected") !== null, {
+      timeout: 60_000,
+    })
+    .then(() => true)
+    .catch(() => false);
+  const status = await page.evaluate(
+    () => document.querySelector(".webrtc-view__status")?.textContent?.trim() ?? "no player rendered",
+  );
+  say(connected, "the camera player reports connected", `status: ${status}`);
+
+  if (connected) {
+    // Waits for a decoded frame, then samples it twice a second apart.
+    const decoded = await page
+      .waitForFunction(
+        () => {
+          const v = document.querySelector<HTMLVideoElement>(".webrtc-view video");
+          return Boolean(v && v.readyState >= 2 && v.videoWidth > 0);
+        },
+        { timeout: 30_000 },
+      )
+      .then(() => true)
+      .catch(() => false);
+
+    const frame = decoded
+      ? await page.evaluate(async () => {
+          const v = document.querySelector<HTMLVideoElement>(".webrtc-view video")!;
+          const t0 = v.currentTime;
+          await new Promise((r) => setTimeout(r, 1500));
+          const canvas = document.createElement("canvas");
+          canvas.width = 64;
+          canvas.height = 36;
+          const ctx = canvas.getContext("2d")!;
+          ctx.drawImage(v, 0, 0, canvas.width, canvas.height);
+          const px = ctx.getImageData(0, 0, canvas.width, canvas.height).data;
+          let sum = 0;
+          let sumSq = 0;
+          const n = px.length / 4;
+          for (let i = 0; i < px.length; i += 4) {
+            const y = 0.299 * px[i] + 0.587 * px[i + 1] + 0.114 * px[i + 2];
+            sum += y;
+            sumSq += y * y;
+          }
+          const mean = sum / n;
+          return {
+            width: v.videoWidth,
+            height: v.videoHeight,
+            advanced: v.currentTime - t0,
+            mean,
+            stdev: Math.sqrt(Math.max(0, sumSq / n - mean * mean)),
+          };
+        })
+      : null;
+
+    say(decoded && frame !== null, "Chrome decoded a video frame", frame ? `${frame.width}x${frame.height}` : "");
+    if (frame) {
+      say(frame.advanced > 0.5, "playback time is advancing, so the stream is live", `+${frame.advanced.toFixed(1)}s`);
+      say(frame.stdev > 2, "the frame has picture content, not black or one flat colour",
+        `mean luma ${frame.mean.toFixed(0)}, stdev ${frame.stdev.toFixed(1)}`);
+    }
+
+    // Read from the browser's own ICE stats, so "relay-only" is observed
+    // rather than inferred from the configuration.
+    const selected = await page.evaluate(async () => {
+      for (const pc of (window as any).__pcs as RTCPeerConnection[]) {
+        const stats = await pc.getStats();
+        const byId = new Map<string, any>();
+        stats.forEach((s) => byId.set(s.id, s));
+        for (const s of byId.values()) {
+          if (s.type === "transport" && s.selectedCandidatePairId) {
+            const pair = byId.get(s.selectedCandidatePairId);
+            const local = pair && byId.get(pair.localCandidateId);
+            if (local) return local.candidateType as string;
+          }
+        }
+      }
+      return null;
+    });
+    say(selected === "relay", "the browser's selected local candidate is a TURN relay", `type: ${selected ?? "none"}`);
+  }
+
+  say(pageErrors.length === 0, "the page raised no uncaught errors", pageErrors.slice(0, 2).join(" | "));
+} finally {
+  await browser.close();
+}
+
+process.exit(failed ? 1 : 0);

scripts/e2e-camera.sh

@@ -42,6 +42,18 @@
 #   GO2RTC_BIN=~/.local/bin/go2rtc
 #   PROXY_DIR=../continuum-proxy
 #   KEEP_DATA=1             leave the seeded farm behind for inspection
+#
+#   BROWSER=1               also drive the real web console in a real Chrome
+#                           (../continuum-app) and check the picture on
+#                           screen. Implies AUTH=supabase, because the app's
+#                           client needs a real Supabase session; needs
+#                           SUPABASE_URL, SUPABASE_SECRET_KEY and
+#                           SUPABASE_PUBLISHABLE_KEY (source ../dev-secrets.sh).
+#                           It creates/reuses one Supabase user,
+#                           continuum-e2e@example.invalid, and resets its
+#                           password each run. Port 3042 must be free.
+#   AUTH=supabase           use that real user's token for the API phases too
+#                           (default is the local dev-auth issuer)
 set -euo pipefail
 
 REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
@@ -64,10 +76,18 @@ R2_ENDPOINT="${R2_ENDPOINT:-http://localhost:9000}"
 export DATABASE_URL VALKEY_URL EDGE_JWT_SECRET PORT
 export R2_ACCESS_KEY_ID R2_SECRET_ACCESS_KEY R2_ENDPOINT
 
+BROWSER="${BROWSER:-0}"
+AUTH="${AUTH:-dev}"
+[ "$BROWSER" = "1" ] && AUTH=supabase
+APP_DIR="${APP_DIR:-$REPO_ROOT/../continuum-app}"
+APP_PORT="${APP_PORT:-3042}"
+CHROME_BIN="${CHROME_BIN:-/usr/bin/google-chrome}"
+
 WORK="$(mktemp -d)"
 BACKEND_PID=""
 PROXY_PID=""
 AUTH_PID=""
+APP_PID=""
 
 fail() { echo "FAIL: $*" >&2; exit 1; }
 ok()   { echo "  ok: $*"; }
@@ -84,6 +104,9 @@ cleanup() {
   for pid in "$PROXY_PID" "$BACKEND_PID" "$AUTH_PID"; do
     [ -n "$pid" ] && kill "$pid" 2>/dev/null || true
   done
+  # The dev server forks workers, so it is signalled as a group: it was started
+  # with setsid, which makes its PID the group id. `kill 0` would hit us.
+  [ -n "$APP_PID" ] && kill -- "-$APP_PID" 2>/dev/null || true
   for _ in $(seq 1 20); do
     [ -n "$PROXY_PID" ] && kill -0 "$PROXY_PID" 2>/dev/null || break
     sleep 0.3
@@ -91,6 +114,7 @@ cleanup() {
   for pid in "$PROXY_PID" "$BACKEND_PID" "$AUTH_PID"; do
     [ -n "$pid" ] && kill -9 "$pid" 2>/dev/null || true
   done
+  [ -n "$APP_PID" ] && kill -9 -- "-$APP_PID" 2>/dev/null || true
   if [ -z "${KEEP_DATA:-}" ] && [ -n "${ORG_ID:-}" ]; then
     psql_q "DELETE FROM organisations WHERE id = '$ORG_ID'" >/dev/null 2>&1 || true
   fi
@@ -137,6 +161,30 @@ else
   ok "could not read $CAMERA_ID's address from printers.toml; skipping the 322 pre-check"
 fi
 
+if [ "$AUTH" = "supabase" ]; then
+  for name in SUPABASE_URL SUPABASE_SECRET_KEY SUPABASE_PUBLISHABLE_KEY; do
+    [ -n "${!name:-}" ] || fail "$name is not set - source ../dev-secrets.sh (AUTH=supabase / BROWSER=1)"
+  done
+fi
+if [ "$BROWSER" = "1" ]; then
+  [ -x "$CHROME_BIN" ] || fail "no Chrome at $CHROME_BIN (set CHROME_BIN)"
+  [ -d "$APP_DIR/apps/web/node_modules" ] || fail "$APP_DIR has no installed web app - run pnpm install there"
+  ss -ltn 2>/dev/null | grep -q ":$APP_PORT " && fail "port $APP_PORT is already in use"
+fi
+
+# The real Supabase user, when one is needed. Captured, never echoed: the JSON
+# holds a password and a live access token.
+if [ "$AUTH" = "supabase" ]; then
+  SB_USER="$(bun scripts/e2e-supabase-user.ts)" || fail "could not provision the Supabase test user"
+  sb_field() { printf '%s' "$SB_USER" | python3 -c 'import json,sys; print(json.load(sys.stdin)[sys.argv[1]])' "$1"; }
+  USER_ID="$(sb_field id)"
+  USER_TOKEN="$(sb_field accessToken)"
+  SB_EMAIL="$(sb_field email)"
+  SB_PASSWORD="$(sb_field password)"
+  unset SB_USER
+  ok "signed in as the real Supabase test user $USER_ID"
+fi
+
 # --- 1. seed ---------------------------------------------------------------
 
 echo "==> seeding an organisation and a farm claiming '$CAMERA_ID'"
@@ -154,14 +202,18 @@ ok "org $ORG_ID, farm $FARM_ID, viewer $USER_ID"
 
 # --- 2. backend ------------------------------------------------------------
 
-echo "==> starting the dev-auth issuer on :$AUTH_PORT and the backend on :$PORT"
-bun scripts/dev-auth.ts --port "$AUTH_PORT" > "$WORK/auth.log" 2>&1 &
-AUTH_PID=$!
-AUTH_URL="http://localhost:$AUTH_PORT"
-for _ in $(seq 1 40); do curl -sf "$AUTH_URL/auth/v1/.well-known/jwks.json" >/dev/null 2>&1 && break; sleep 0.25; done
-curl -sf "$AUTH_URL/auth/v1/.well-known/jwks.json" >/dev/null 2>&1 || fail "dev-auth did not come up"
-export SUPABASE_URL="$AUTH_URL"
-USER_TOKEN="$(curl -sf "$AUTH_URL/token?sub=$USER_ID" | python3 -c 'import json,sys; print(json.load(sys.stdin)["access_token"])')"
+if [ "$AUTH" = "dev" ]; then
+  echo "==> starting the dev-auth issuer on :$AUTH_PORT and the backend on :$PORT"
+  bun scripts/dev-auth.ts --port "$AUTH_PORT" > "$WORK/auth.log" 2>&1 &
+  AUTH_PID=$!
+  AUTH_URL="http://localhost:$AUTH_PORT"
+  for _ in $(seq 1 40); do curl -sf "$AUTH_URL/auth/v1/.well-known/jwks.json" >/dev/null 2>&1 && break; sleep 0.25; done
+  curl -sf "$AUTH_URL/auth/v1/.well-known/jwks.json" >/dev/null 2>&1 || fail "dev-auth did not come up"
+  export SUPABASE_URL="$AUTH_URL"
+  USER_TOKEN="$(curl -sf "$AUTH_URL/token?sub=$USER_ID" | python3 -c 'import json,sys; print(json.load(sys.stdin)["access_token"])')"
+else
+  echo "==> starting the backend on :$PORT, verifying tokens against the real Supabase JWKS"
+fi
 bun run src/index.ts > "$WORK/backend.log" 2>&1 &
 BACKEND_PID=$!
 for _ in $(seq 1 60); do curl -sf "$BASE/health" >/dev/null 2>&1 && break; sleep 0.5; done
@@ -251,9 +303,57 @@ else
   echo "           The TURN-relayed path, which the browser is pinned to, is NOT verified by this run."
 fi
 
+# --- 5. the real web console in a real browser -------------------------------
+
+if [ "$BROWSER" = "1" ]; then
+  echo "==> starting the web console on :$APP_PORT"
+  # setsid so the whole dev server (it forks workers) shares one process group
+  # that cleanup can signal; the app is told where the backend is and given the
+  # real Supabase project it signs in against.
+  setsid env -C "$APP_DIR/apps/web" \
+    NUXT_PUBLIC_API_BASE="$BASE" \
+    NUXT_PUBLIC_WS_BASE="ws://localhost:$PORT" \
+    SUPABASE_URL="$SUPABASE_URL" \
+    SUPABASE_KEY="$SUPABASE_PUBLISHABLE_KEY" \
+    pnpm exec nuxt dev --host 127.0.0.1 --port "$APP_PORT" > "$WORK/app.log" 2>&1 &
+  APP_PID=$!
+  for _ in $(seq 1 120); do
+    curl -sf -o /dev/null "http://127.0.0.1:$APP_PORT/login" 2>/dev/null && break
+    sleep 1
+  done
+  curl -sf -o /dev/null "http://127.0.0.1:$APP_PORT/login" \
+    || { tail -20 "$WORK/app.log" >&2; fail "the web console did not start on :$APP_PORT"; }
+  ok "the web console is up"
+
+  echo "==> the real console in real Chrome: sign in, open the fleet, watch the camera"
+  BROWSER_OUT="$(env APP_URL="http://127.0.0.1:$APP_PORT" SUPABASE_E2E_EMAIL="$SB_EMAIL" \
+    SUPABASE_E2E_PASSWORD="$SB_PASSWORD" CHROME_BIN="$CHROME_BIN" \
+    timeout 240 bun scripts/e2e-camera-browser.ts 2>&1 || true)"
+  seen=0; bad=0
+  while IFS=$'\t' read -r verdict label; do
+    case "$verdict" in
+      ok)     ok "$label"; seen=$((seen + 1)) ;;
+      FAILED) echo "  FAILED: $label" >&2; seen=$((seen + 1)); bad=$((bad + 1)) ;;
+    esac
+  done < <(printf '%s\n' "$BROWSER_OUT" | grep -E "^(ok|FAILED)$(printf '\t')")
+  # A complete run is nine results. Fewer means the driver died part way, and a
+  # silent driver must not read as a pass.
+  if [ "$seen" -lt 9 ] && [ "$bad" -eq 0 ]; then
+    bad=1; echo "  FAILED: the browser run produced only $seen of 9 results" >&2
+  fi
+  if [ "$bad" -ne 0 ]; then
+    printf '%s\n' "$BROWSER_OUT" | sed 's/^/      | /' >&2
+    echo "--- web console log (tail) ---" >&2; tail -15 "$WORK/app.log" >&2
+    fail "the real browser did not show the camera"
+  fi
+fi
+
 echo
 echo "PASS - a real WebRTC peer received video from the printer's camera"
 echo "       through the backend relay, the uplink, the proxy and go2rtc."
 if [ -n "${CLOUDFLARE_TURN_KEY_ID:-}" ] && [ -n "${CLOUDFLARE_TURN_API_TOKEN:-}" ]; then
   echo "       Both directly and pinned to relay-only through Cloudflare TURN."
 fi
+if [ "$BROWSER" = "1" ]; then
+  echo "       And the real web console, in real Chrome, showed the picture."
+fi

scripts/e2e-supabase-user.ts

@@ -0,0 +1,54 @@
+/**
+ * Provisions the real Supabase test user for scripts/e2e-camera.sh and prints
+ * one JSON object: { id, email, password, accessToken }.
+ *
+ * The output carries a password and a live access token, so the caller must
+ * capture it into a variable and never echo it. The account is reused rather
+ * than piling up one per run: `.invalid` is reserved by RFC 2606, so the
+ * address can never belong to a real person, and its password is reset every
+ * time to a fresh random one that exists only for this run.
+ *
+ * Environment: SUPABASE_URL, SUPABASE_SECRET_KEY, SUPABASE_PUBLISHABLE_KEY.
+ */
+const url = process.env.SUPABASE_URL!;
+const secret = process.env.SUPABASE_SECRET_KEY!;
+const publishable = process.env.SUPABASE_PUBLISHABLE_KEY!;
+const email = "continuum-e2e@example.invalid";
+const password = `e2e-${crypto.randomUUID()}`;
+const admin = { apikey: secret, Authorization: `Bearer ${secret}`, "Content-Type": "application/json" };
+
+const die = (message: string): never => {
+  console.error(`e2e-supabase-user: ${message}`);
+  process.exit(1);
+};
+
+let id: string | undefined;
+const made = await fetch(`${url}/auth/v1/admin/users`, {
+  method: "POST",
+  headers: admin,
+  body: JSON.stringify({ email, password, email_confirm: true }),
+});
+if (made.ok) {
+  id = ((await made.json()) as { id?: string }).id;
+} else {
+  const list = (await fetch(`${url}/auth/v1/admin/users?per_page=200`, { headers: admin }).then((r) => r.json())) as {
+    users?: { id: string; email: string }[];
+  };
+  id = list.users?.find((u) => u.email === email)?.id;
+  if (!id) die(`could not create or find the test user (HTTP ${made.status})`);
+  const reset = await fetch(`${url}/auth/v1/admin/users/${id}`, {
+    method: "PUT",
+    headers: admin,
+    body: JSON.stringify({ password, email_confirm: true }),
+  });
+  if (!reset.ok) die(`could not reset the test user's password (HTTP ${reset.status})`);
+}
+
+const session = (await fetch(`${url}/auth/v1/token?grant_type=password`, {
+  method: "POST",
+  headers: { apikey: publishable, "Content-Type": "application/json" },
+  body: JSON.stringify({ email, password }),
+}).then((r) => r.json())) as { access_token?: string };
+if (!session.access_token || !id) die("the test user could not sign in");
+
+console.log(JSON.stringify({ id, email, password, accessToken: session.access_token }));