Add a runnable dev environment, and fix the pins that stopped it booting
The edge route had never been run. Adding a way to run it turned up that the server could not start at all, for two independent dependency reasons that typechecking could not see: - `elysia: ^1.1.26` had resolved to 1.4.30, which calls `t.Module` - a TypeBox 0.34 API - while this repo pinned TypeBox 0.33. `t.Module is not a function`. - `drizzle-typebox: ^0.2.1` resolved to a version whose peers demand `drizzle-orm >= 0.36`, against the `^0.33.0` pinned here. Importing it threw `Export named 'getViewSelectedFields' not found`. Both are caret ranges that drifted into combinations nobody chose. elysia, @sinclair/typebox and the drizzle packages are now pinned tightly, TypeBox forward to 0.34.52 (which satisfies Elysia and drizzle-typebox 0.1.1 alike), and tests/dependency_identity.test.ts fails if the TypeBox version ever differs between here and @continuum/ts-types - the skew that made every shared schema invalid, with an error message pointing at Standard Schema instead. Dropping to drizzle-typebox 0.1.1 widens `jsonb` in the generated validators, so `metadata` needs an explicit object shape in validators.ts; the column is declared `$type<Record<string, unknown>>()` and a validated body otherwise is not assignable to `.values()`. scripts/: - `install-docker.sh` - one-time host setup, kept separate because it touches the machine rather than containers. - `dev-env.sh up|down|reset|status|psql|schema|env` - Valkey + Postgres/pgvector. Uses docker-compose.yml when a daemon answers, and otherwise runs the same two services from apt. That fallback is not hypothetical: Docker cannot run in this sandbox at all. dockerd needs CAP_NET_ADMIN to build its bridge; with `--bridge=none --iptables=false` it starts but cannot unpack an image without CAP_SYS_ADMIN; with `--storage-driver=vfs` it still fails because registering a layer needs user namespaces, which are blocked outright. All three are host-level grants, so no script can work around them - hence a second path rather than a broken first one. - `e2e-edge.sh` - the test the unit tests cannot be. Seeds a farm, claims its printers, mints an edge token, runs the real continuum-proxy, and asserts live telemetry reached Postgres and a command reached a printer. It only sends a fan speed, only to a printer reporting IDLE, and puts it back afterwards. - `capture-gateway-frames.js` - records what a gateway really sends, for refreshing tests/fixtures/. Read-only unless FAN_TEST_PRINTER is set. The test comment referred to this before it existed. Verified end to end, from a dropped and freshly migrated database: `h2c` persisted as printing/RUNNING 22% layer 2/25 nozzle 230 bed 35 with 3 HMS alerts, `p1p` as idle/IDLE, and `POST /printers/:id/commands` reached the P1P's fan and came back ok:true. An unclaimed printer is refused 409 PRINTER_UNREACHABLE rather than guessed at. Two things the e2e script got wrong first, both now fixed in it: it waited for the first printer row instead of all of them, which silently tested only the v2 printer (a v1 Bambu reports one tick later); and `$!` after `( cd ... ) &` is the subshell, so cleanup left an orphaned gateway reconnecting with a token for a deleted farm and polluting the next run. Note: `drizzle/` is gitignored, so the generated migration is not committed and there is no migration history in the repo. Left as-is deliberately - that is a workflow call, flagged in PLAN.md.
This commit is contained in:
Executable
+225
@@ -0,0 +1,225 @@
|
||||
#!/usr/bin/env bash
|
||||
# Brings the local dependency stack (Valkey + Postgres/pgvector) up and down,
|
||||
# and applies the Drizzle schema to it.
|
||||
#
|
||||
# Two backends behind one interface, chosen automatically:
|
||||
#
|
||||
# docker docker-compose.yml - Valkey 8 + pgvector/pgvector:pg17. What CI
|
||||
# and a normal dev machine should use; closest to production.
|
||||
# native apt packages started directly - postgresql-16 + pgvector + Valkey
|
||||
# 7.2 from Ubuntu's own repositories.
|
||||
#
|
||||
# The native path exists because Docker provably cannot run in some sandboxes,
|
||||
# including the one this was written in: dockerd needs CAP_NET_ADMIN to build
|
||||
# its bridge, and even with `--bridge=none --iptables=false
|
||||
# --storage-driver=vfs` it cannot unpack an image, because registering a layer
|
||||
# needs user namespaces. Both are host-level grants, so there is nothing a
|
||||
# script can do about it. Rather than leave the database untestable there, this
|
||||
# falls back to running the same two services from apt. Set CONTINUUM_DEV_BACKEND
|
||||
# to force one.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/dev-env.sh up start the services, wait until they answer
|
||||
# scripts/dev-env.sh down stop them (keeps data)
|
||||
# scripts/dev-env.sh reset stop them AND delete all data
|
||||
# scripts/dev-env.sh status what is running, and which backend
|
||||
# scripts/dev-env.sh psql open a shell on the database
|
||||
# scripts/dev-env.sh schema apply the Drizzle schema (drizzle-kit push)
|
||||
# scripts/dev-env.sh env print the DATABASE_URL/VALKEY_URL to use
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$REPO_ROOT"
|
||||
|
||||
PG_VERSION=16
|
||||
PG_PORT=5432
|
||||
VALKEY_PORT=6379
|
||||
# Straight at Postgres, not pgCat on 6432: .env.example points at the pooler for
|
||||
# production, and neither backend here runs one.
|
||||
DEV_DATABASE_URL="postgres://continuum:continuum@localhost:${PG_PORT}/continuum"
|
||||
DEV_VALKEY_URL="redis://localhost:${VALKEY_PORT}"
|
||||
|
||||
NATIVE_STATE="${XDG_RUNTIME_DIR:-/tmp}/continuum-dev-env"
|
||||
|
||||
# --- backend selection -----------------------------------------------------
|
||||
|
||||
# Docker is used when its daemon is already answering. Deliberately does not
|
||||
# try to start dockerd: on a machine where it works there is a service manager
|
||||
# doing that, and on a machine where it does not, starting it just produces a
|
||||
# daemon that cannot unpack an image (see the header). Force either backend
|
||||
# with CONTINUUM_DEV_BACKEND=docker|native.
|
||||
docker_available() {
|
||||
command -v docker >/dev/null 2>&1 || return 1
|
||||
sudo -n docker info >/dev/null 2>&1
|
||||
}
|
||||
|
||||
backend() {
|
||||
if [ -n "${CONTINUUM_DEV_BACKEND:-}" ]; then
|
||||
echo "$CONTINUUM_DEV_BACKEND"
|
||||
elif docker_available; then
|
||||
echo docker
|
||||
else
|
||||
echo native
|
||||
fi
|
||||
}
|
||||
|
||||
# --- waiting ---------------------------------------------------------------
|
||||
|
||||
# Poll for readiness rather than sleeping a fixed amount: Postgres accepts
|
||||
# connections before it will serve them, and a failure should report why.
|
||||
wait_for_postgres() {
|
||||
for _ in $(seq 1 60); do
|
||||
if pg_isready -h localhost -p "$PG_PORT" -U continuum -d continuum >/dev/null 2>&1; then
|
||||
return 0
|
||||
fi
|
||||
sleep 0.5
|
||||
done
|
||||
echo "postgres did not become ready on port $PG_PORT" >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
wait_for_valkey() {
|
||||
for _ in $(seq 1 40); do
|
||||
if valkey-cli -p "$VALKEY_PORT" ping 2>/dev/null | grep -q PONG; then
|
||||
return 0
|
||||
fi
|
||||
sleep 0.5
|
||||
done
|
||||
echo "valkey did not answer on port $VALKEY_PORT" >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
# --- native backend --------------------------------------------------------
|
||||
|
||||
native_up() {
|
||||
mkdir -p "$NATIVE_STATE"
|
||||
|
||||
if ! pg_isready -h localhost -p "$PG_PORT" >/dev/null 2>&1; then
|
||||
echo "==> starting postgresql-$PG_VERSION"
|
||||
# pg_ctlcluster rather than systemctl: there is no service manager here.
|
||||
sudo pg_ctlcluster "$PG_VERSION" main start
|
||||
fi
|
||||
wait_for_postgres_socket
|
||||
|
||||
# Idempotent: both of these are no-ops on a second run.
|
||||
if ! sudo -u postgres psql -tAc "SELECT 1 FROM pg_roles WHERE rolname='continuum'" | grep -q 1; then
|
||||
echo "==> creating role 'continuum'"
|
||||
sudo -u postgres psql -qc "CREATE ROLE continuum LOGIN PASSWORD 'continuum' SUPERUSER" >/dev/null
|
||||
fi
|
||||
if ! sudo -u postgres psql -tAc "SELECT 1 FROM pg_database WHERE datname='continuum'" | grep -q 1; then
|
||||
echo "==> creating database 'continuum'"
|
||||
sudo -u postgres createdb -O continuum continuum
|
||||
fi
|
||||
# frame_embeddings needs it, and it is why the compose file uses the
|
||||
# pgvector image rather than plain postgres.
|
||||
sudo -u postgres psql -d continuum -qc "CREATE EXTENSION IF NOT EXISTS vector" >/dev/null
|
||||
|
||||
if ! valkey-cli -p "$VALKEY_PORT" ping >/dev/null 2>&1; then
|
||||
echo "==> starting valkey"
|
||||
valkey-server --port "$VALKEY_PORT" --daemonize yes \
|
||||
--dir "$NATIVE_STATE" --appendonly yes \
|
||||
--logfile "$NATIVE_STATE/valkey.log"
|
||||
fi
|
||||
wait_for_valkey
|
||||
}
|
||||
|
||||
# Postgres has to be up before the role exists, so this waits on the server
|
||||
# rather than on a successful `continuum` login.
|
||||
wait_for_postgres_socket() {
|
||||
for _ in $(seq 1 60); do
|
||||
if sudo -u postgres pg_isready -q 2>/dev/null; then return 0; fi
|
||||
sleep 0.5
|
||||
done
|
||||
echo "postgres did not start" >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
native_down() {
|
||||
if valkey-cli -p "$VALKEY_PORT" ping >/dev/null 2>&1; then
|
||||
echo "==> stopping valkey"
|
||||
valkey-cli -p "$VALKEY_PORT" shutdown nosave 2>/dev/null || true
|
||||
fi
|
||||
if sudo -u postgres pg_isready -q 2>/dev/null; then
|
||||
echo "==> stopping postgresql-$PG_VERSION"
|
||||
sudo pg_ctlcluster "$PG_VERSION" main stop || true
|
||||
fi
|
||||
}
|
||||
|
||||
native_reset() {
|
||||
native_down
|
||||
echo "==> dropping the continuum database and valkey's data"
|
||||
sudo pg_ctlcluster "$PG_VERSION" main start
|
||||
wait_for_postgres_socket
|
||||
sudo -u postgres dropdb --if-exists continuum
|
||||
sudo pg_ctlcluster "$PG_VERSION" main stop || true
|
||||
rm -rf "$NATIVE_STATE"
|
||||
}
|
||||
|
||||
native_status() {
|
||||
echo "backend: native (apt packages, no container runtime)"
|
||||
if sudo -u postgres pg_isready -q 2>/dev/null; then
|
||||
echo " postgres: up on :$PG_PORT ($(sudo -u postgres psql -tAc 'SELECT version()' | cut -d, -f1))"
|
||||
else
|
||||
echo " postgres: down"
|
||||
fi
|
||||
if valkey-cli -p "$VALKEY_PORT" ping >/dev/null 2>&1; then
|
||||
echo " valkey: up on :$VALKEY_PORT"
|
||||
else
|
||||
echo " valkey: down"
|
||||
fi
|
||||
}
|
||||
|
||||
# --- docker backend --------------------------------------------------------
|
||||
|
||||
docker_up() {
|
||||
sudo docker compose up -d
|
||||
echo "==> waiting for containers to report healthy"
|
||||
for _ in $(seq 1 60); do
|
||||
local unhealthy
|
||||
unhealthy="$(sudo docker compose ps --format '{{.Service}} {{.Health}}' \
|
||||
| awk '$2 != "healthy" { print $1 }' || true)"
|
||||
[ -z "$unhealthy" ] && return 0
|
||||
sleep 1
|
||||
done
|
||||
echo "timed out: $(sudo docker compose ps --format '{{.Service}} {{.Health}}')" >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
# --- commands --------------------------------------------------------------
|
||||
|
||||
cmd_up() {
|
||||
case "$(backend)" in
|
||||
docker) echo "==> backend: docker compose"; docker_up ;;
|
||||
native) echo "==> backend: native (docker is unavailable here - see this script's header)"; native_up ;;
|
||||
esac
|
||||
echo
|
||||
cmd_env
|
||||
echo
|
||||
echo "Next: scripts/dev-env.sh schema"
|
||||
}
|
||||
|
||||
cmd_down() { case "$(backend)" in docker) sudo docker compose down ;; native) native_down ;; esac; }
|
||||
cmd_reset() { case "$(backend)" in docker) sudo docker compose down -v ;; native) native_reset ;; esac; }
|
||||
cmd_status() { case "$(backend)" in docker) echo "backend: docker"; sudo docker compose ps ;; native) native_status ;; esac; }
|
||||
cmd_psql() { PGPASSWORD=continuum psql -h localhost -p "$PG_PORT" -U continuum -d continuum "$@"; }
|
||||
cmd_env() { echo "DATABASE_URL=$DEV_DATABASE_URL"; echo "VALKEY_URL=$DEV_VALKEY_URL"; }
|
||||
|
||||
cmd_schema() {
|
||||
# `push` rather than generate+migrate: no migrations are committed and the
|
||||
# schema is still moving. Switch once it settles - push diffs the live
|
||||
# database against the schema and will happily drop a column, which is fine
|
||||
# locally and nowhere else.
|
||||
echo "==> applying the Drizzle schema to $DEV_DATABASE_URL"
|
||||
DATABASE_URL="$DEV_DATABASE_URL" bunx drizzle-kit push --force
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
up) cmd_up ;;
|
||||
down) cmd_down ;;
|
||||
reset) cmd_reset ;;
|
||||
status) cmd_status ;;
|
||||
psql) shift; cmd_psql "$@" ;;
|
||||
schema) cmd_schema ;;
|
||||
env) cmd_env ;;
|
||||
*) sed -n '2,30p' "${BASH_SOURCE[0]}" | sed 's/^# \?//'; exit 1 ;;
|
||||
esac
|
||||
Reference in New Issue
Block a user