octoturge
7b85f4dff1
The whole point of driving a real browser instead of curling api/frontend separately: none of this session's prior "live verification" ever exercised the same-origin routing production relies on (Caddy: /ws*+/api/* -> api, else -> frontend), so a real browser's relative fetch()/WebSocket calls were never actually proven to resolve. e2e/proxy.ts mirrors that routing (no caddy binary available locally); global-setup.ts/global-teardown.ts orchestrate throwaway infra + seeded data + the api/frontend/proxy processes end to end. Getting the suite green surfaced two genuine bugs invisible to unit tests: - index.pug loaded map.js via two <script type="module"> tags (one moved to <head> to fix load-order, the original left in place by mistake), causing Alpine's x-init="init()" to run twice and Leaflet to throw "Map container is already initialized" on the second call. - map.js's exportRegion() passed the Alpine-reactive `regionBounds` object straight into worker.postMessage(); Alpine wraps assigned state in Proxies, which the structured clone algorithm can't clone, so every export silently failed. Fixed by spreading into a plain object first. Covers the two flows flagged all session as verified only at the unit/curl level: the marker click-to-place/edit popup (including that a marker created while linked shows up in a second browser context with the same session, proving server-side sync) and the region-select drag + glTF export (including a real triggered file download).
134 lines
6.8 KiB
Markdown
134 lines
6.8 KiB
Markdown
# MCMapper-Backend
|
|
|
|
Map-render backend for [MCMapper-Mod](https://git.octoturge.com/octoturge/MCMapper-Mod). Does
|
|
all the heavy lifting a thin in-game mod shouldn't: persists world state, renders 2D tiles and
|
|
3D meshes, relays chat, and serves the web map viewer — instead of the MC server itself burning
|
|
CPU/RAM on rendering the way Bluemap/Dynmap do.
|
|
|
|
## Services
|
|
|
|
Three independently-deployable services, each its own docker-compose service:
|
|
|
|
- `api/` — ElysiaJS on Bun. The I/O layer: WS gateway for mod connections, chat relay, chunk
|
|
store, marker/waypoint sharing, admin config, region export, tile/mesh serving.
|
|
- `worker/` — Rust. CPU-bound rendering: tile rasterization and chunk meshing, consumed off a
|
|
Redis dirty-chunk stream. Stateless — scale it with `docker compose up --scale worker=N`, or
|
|
run instances on separate hardware pointed at the same Postgres/Redis/MinIO over a private
|
|
network. Rendering strategy (`cpu`/`gpu`/`hybrid`) is config-selectable behind a
|
|
`RenderBackend` trait; only `cpu` (rayon) exists so far — `gpu`/`hybrid` (wgpu) land in
|
|
Phase 8.
|
|
- `frontend/` — ElysiaJS + Pug + Tailwind 4 + Alpine.js. The public-facing pages (map viewer,
|
|
chat, admin panel). Stateless — no DB access, calls `api` for anything server-rendered; the
|
|
browser's own live map/chat/tile traffic talks to `api` directly, not proxied through here.
|
|
|
|
Plus `postgres` (source chunk data, accounts, chat history, config, render-artifact metadata
|
|
pointers), `redis` (dirty-chunk queue, pub/sub, link-code TTLs), and `caddy` (reverse proxy:
|
|
`/ws` + `/api/*` → `api`, everything else → `frontend`).
|
|
|
|
Postgres/Redis are never exposed publicly — only reachable on the compose network or a private
|
|
network (VPN/Tailscale/LAN) for remote `worker` instances.
|
|
|
|
### Object storage
|
|
|
|
Rendered tile PNGs and mesh binaries live in a `mcmapper-tiles` bucket on an existing shared
|
|
MinIO instance (`devstack-minio` on octo-winsrv) rather than a per-stack `minio` container —
|
|
kept out of Postgres so backups stay free of large binaries, and any `worker` instance (local or
|
|
remote) has a shared place to write output. `api`/`worker` authenticate with a dedicated
|
|
`mcmapper` access key whose policy (`mcmapper-tiles-rw`) only grants
|
|
`GetObject`/`PutObject`/`DeleteObject`/`ListBucket` on that one bucket — it can't see or touch
|
|
anything else on the shared instance. Provisioned via:
|
|
|
|
```
|
|
mc mb myminio/mcmapper-tiles
|
|
mc admin policy create myminio mcmapper-tiles-rw mcmapper-policy.json # see git history for the policy JSON
|
|
mc admin user add myminio mcmapper <generated secret>
|
|
mc admin policy attach myminio mcmapper-tiles-rw --user mcmapper
|
|
```
|
|
|
|
`api/.env.example`/`worker/.env.example` point `MINIO_ENDPOINT`/`MINIO_PORT` at that instance's
|
|
LAN address; swap to the public gateway (`s3.octoturge.com:443`, `MINIO_USE_SSL=true`) if a
|
|
stack isn't on the same LAN. `MINIO_SECRET_KEY` is a real credential and is deliberately **not**
|
|
committed — set it in an untracked `./api/.env` / `./worker/.env` (docker-compose layers those on
|
|
top of the tracked `.env.example`, see `docker-compose.yml`).
|
|
|
|
## Running
|
|
|
|
```
|
|
docker compose up
|
|
```
|
|
|
|
`api` on :3000, `frontend` on :3001, both behind Caddy on :80. `api` applies its Postgres
|
|
migrations on startup (see `api/src/db/migrate.ts`); `worker` consumes the `mcmapper:dirty-chunks`
|
|
Redis stream via a consumer group (`mcmapper-workers`) so multiple instances split work safely.
|
|
|
|
### Phase 1: connecting a mod instance
|
|
|
|
There's no admin registration API yet (Phase 6) — seed one server row by hand, then point the
|
|
mod's `MapperConfig` at the same token:
|
|
|
|
```
|
|
docker compose run --rm api bun run seed
|
|
```
|
|
|
|
reads `MCMAPPER_SEED_SERVER_NAME`/`MCMAPPER_SEED_SERVER_TOKEN` from `api/.env.example` (edit
|
|
those first, or override with `-e`). Once the mod connects and sends its initial chunk backfill,
|
|
tiles appear at `GET /api/tiles/:serverId/:dimension/:zoom/:tileX/:tileY.png` (zoom is always `0`
|
|
for now — see `worker/src/render/cpu.rs`) and the frontend's Leaflet viewer picks them up
|
|
automatically from `GET /api/servers`.
|
|
|
|
## Running tests
|
|
|
|
`worker`'s tests (`cargo test`, in `worker/`) are pure unit tests (greedy mesher, tile
|
|
rasterizer, block-color palette) and need nothing running. `api`'s and `frontend`'s (`bun test`,
|
|
in each directory) are integration tests against real infra — start it first:
|
|
|
|
```
|
|
docker run -d --name mcmapper-test-pg -e POSTGRES_USER=mcmapper -e POSTGRES_PASSWORD=mcmapper -e POSTGRES_DB=mcmapper -p 15432:5432 postgres:17-alpine
|
|
docker run -d --name mcmapper-test-redis -p 16379:6379 redis:7-alpine
|
|
docker run -d --name mcmapper-test-minio -e MINIO_ROOT_USER=mcmapper -e MINIO_ROOT_PASSWORD=mcmapper-dev-only -p 19000:9000 minio/minio:latest server /data
|
|
cd api && DATABASE_URL=postgres://mcmapper:mcmapper@localhost:15432/mcmapper bun run migrate
|
|
```
|
|
|
|
then, from `api/` or `frontend/`:
|
|
|
|
```
|
|
DATABASE_URL=postgres://mcmapper:mcmapper@localhost:15432/mcmapper \
|
|
REDIS_URL=redis://localhost:16379 \
|
|
MINIO_ENDPOINT=localhost MINIO_PORT=19000 MINIO_ACCESS_KEY=mcmapper MINIO_SECRET_KEY=mcmapper-dev-only \
|
|
bun test
|
|
```
|
|
|
|
Each test file creates and tears down its own server row (random token per run) so runs never
|
|
collide with each other or with real dev data — see `api/src/test-helpers.ts`.
|
|
|
|
## Running the e2e suite
|
|
|
|
`bun test` above exercises `api` and `frontend` independently — it never proves the browser can
|
|
actually reach both through the same origin the way production's Caddy routing does (`/ws*` and
|
|
`/api/*` -> `api`, everything else -> `frontend`, see `Caddyfile`). `e2e/` is a standing
|
|
Playwright suite that closes that gap: it drives a real Chromium browser against the full stack
|
|
behind a small routing-equivalent proxy (`e2e/proxy.ts` — no `caddy` binary is available in this
|
|
dev environment, so it isn't real Caddy, just the same three routing rules).
|
|
|
|
```
|
|
cd e2e
|
|
bun install
|
|
bunx playwright install chromium # one-time, downloads the browser binary
|
|
bunx playwright test
|
|
```
|
|
|
|
`global-setup.ts` does everything by itself — no manual container/migration steps needed first
|
|
(unlike the `bun test` section above): throwaway Postgres/Redis/MinIO containers
|
|
(`mcmapper-e2e-*`, distinct names/ports from the `bun test` ones so both can run at once),
|
|
migrations, a seeded server + linked account/session + a 5x5-chunk terrain footprint around the
|
|
world origin, then the `api`/`frontend`/proxy processes. `global-teardown.ts` kills every spawned
|
|
process and removes the containers afterward. Covers the two UI flows most worth a real
|
|
click-through: the marker click-to-place/edit popup (`tests/markers.spec.ts`, including that a
|
|
marker created while linked shows up in a second browser context with the same session — the
|
|
cross-device sync claim) and the region-select drag + glTF export (`tests/region-export.spec.ts`,
|
|
including a real triggered file download).
|
|
|
|
## Attribution
|
|
|
|
See `THIRD_PARTY_NOTICES.md`.
|