octoturge
dc7185c15e
Requested after Phase 2: from here on, MCMapper development follows TDD (test-first) — this retrofits the pieces already built before that request landed. worker: unit tests for the tile rasterizer (background fill, exact upscaled-block boundaries, full-grid painting, out-of-bounds columns) and the block-color palette (distinctness checks, including that the "unmapped block" placeholder never accidentally collides with a real block's color). 16 tests total alongside the existing mesher tests. api: wired up `bun test`. Unit tests for chunkOf's coordinate math. Integration tests (real Postgres/Redis/MinIO, see README's new "Running tests" section) for wsGateway.message() — auth accept/reject, upsert + dedup on columns/sections, not-authenticated/invalid-JSON handling — and for the tile/mesh/servers HTTP routes, driven through Elysia's in-process `.handle()` rather than a bound port (sidesteps the stale dev-server port-collision issue hit repeatedly this session). index.ts now exports `app` and only calls `.listen()` when run directly, specifically so tests can drive it this way. frontend: extracted mesh.js's binary-format parser into its own ESM module (mesh-format.js) so it's unit-testable without a browser/Babylon; mesh.js now imports it. Tests build a buffer independently of the parser (mirroring worker's encoder layout) so a mismatch in either direction — Rust producer or JS consumer drifting — would be caught.
107 lines
5.2 KiB
Markdown
107 lines
5.2 KiB
Markdown
# MCMapper-Backend
|
|
|
|
Map-render backend for [MCMapper-Mod](https://git.octoturge.com/octoturge/MCMapper-Mod). Does
|
|
all the heavy lifting a thin in-game mod shouldn't: persists world state, renders 2D tiles and
|
|
3D meshes, relays chat, and serves the web map viewer — instead of the MC server itself burning
|
|
CPU/RAM on rendering the way Bluemap/Dynmap do.
|
|
|
|
## Services
|
|
|
|
Three independently-deployable services, each its own docker-compose service:
|
|
|
|
- `api/` — ElysiaJS on Bun. The I/O layer: WS gateway for mod connections, chat relay, chunk
|
|
store, marker/waypoint sharing, admin config, region export, tile/mesh serving.
|
|
- `worker/` — Rust. CPU-bound rendering: tile rasterization and chunk meshing, consumed off a
|
|
Redis dirty-chunk stream. Stateless — scale it with `docker compose up --scale worker=N`, or
|
|
run instances on separate hardware pointed at the same Postgres/Redis/MinIO over a private
|
|
network. Rendering strategy (`cpu`/`gpu`/`hybrid`) is config-selectable behind a
|
|
`RenderBackend` trait; only `cpu` (rayon) exists so far — `gpu`/`hybrid` (wgpu) land in
|
|
Phase 8.
|
|
- `frontend/` — ElysiaJS + Pug + Tailwind 4 + Alpine.js. The public-facing pages (map viewer,
|
|
chat, admin panel). Stateless — no DB access, calls `api` for anything server-rendered; the
|
|
browser's own live map/chat/tile traffic talks to `api` directly, not proxied through here.
|
|
|
|
Plus `postgres` (source chunk data, accounts, chat history, config, render-artifact metadata
|
|
pointers), `redis` (dirty-chunk queue, pub/sub, link-code TTLs), and `caddy` (reverse proxy:
|
|
`/ws` + `/api/*` → `api`, everything else → `frontend`).
|
|
|
|
Postgres/Redis are never exposed publicly — only reachable on the compose network or a private
|
|
network (VPN/Tailscale/LAN) for remote `worker` instances.
|
|
|
|
### Object storage
|
|
|
|
Rendered tile PNGs and mesh binaries live in a `mcmapper-tiles` bucket on an existing shared
|
|
MinIO instance (`devstack-minio` on octo-winsrv) rather than a per-stack `minio` container —
|
|
kept out of Postgres so backups stay free of large binaries, and any `worker` instance (local or
|
|
remote) has a shared place to write output. `api`/`worker` authenticate with a dedicated
|
|
`mcmapper` access key whose policy (`mcmapper-tiles-rw`) only grants
|
|
`GetObject`/`PutObject`/`DeleteObject`/`ListBucket` on that one bucket — it can't see or touch
|
|
anything else on the shared instance. Provisioned via:
|
|
|
|
```
|
|
mc mb myminio/mcmapper-tiles
|
|
mc admin policy create myminio mcmapper-tiles-rw mcmapper-policy.json # see git history for the policy JSON
|
|
mc admin user add myminio mcmapper <generated secret>
|
|
mc admin policy attach myminio mcmapper-tiles-rw --user mcmapper
|
|
```
|
|
|
|
`api/.env.example`/`worker/.env.example` point `MINIO_ENDPOINT`/`MINIO_PORT` at that instance's
|
|
LAN address; swap to the public gateway (`s3.octoturge.com:443`, `MINIO_USE_SSL=true`) if a
|
|
stack isn't on the same LAN. `MINIO_SECRET_KEY` is a real credential and is deliberately **not**
|
|
committed — set it in an untracked `./api/.env` / `./worker/.env` (docker-compose layers those on
|
|
top of the tracked `.env.example`, see `docker-compose.yml`).
|
|
|
|
## Running
|
|
|
|
```
|
|
docker compose up
|
|
```
|
|
|
|
`api` on :3000, `frontend` on :3001, both behind Caddy on :80. `api` applies its Postgres
|
|
migrations on startup (see `api/src/db/migrate.ts`); `worker` consumes the `mcmapper:dirty-chunks`
|
|
Redis stream via a consumer group (`mcmapper-workers`) so multiple instances split work safely.
|
|
|
|
### Phase 1: connecting a mod instance
|
|
|
|
There's no admin registration API yet (Phase 6) — seed one server row by hand, then point the
|
|
mod's `MapperConfig` at the same token:
|
|
|
|
```
|
|
docker compose run --rm api bun run seed
|
|
```
|
|
|
|
reads `MCMAPPER_SEED_SERVER_NAME`/`MCMAPPER_SEED_SERVER_TOKEN` from `api/.env.example` (edit
|
|
those first, or override with `-e`). Once the mod connects and sends its initial chunk backfill,
|
|
tiles appear at `GET /api/tiles/:serverId/:dimension/:zoom/:tileX/:tileY.png` (zoom is always `0`
|
|
for now — see `worker/src/render/cpu.rs`) and the frontend's Leaflet viewer picks them up
|
|
automatically from `GET /api/servers`.
|
|
|
|
## Running tests
|
|
|
|
`worker`'s tests (`cargo test`, in `worker/`) are pure unit tests (greedy mesher, tile
|
|
rasterizer, block-color palette) and need nothing running. `api`'s and `frontend`'s (`bun test`,
|
|
in each directory) are integration tests against real infra — start it first:
|
|
|
|
```
|
|
docker run -d --name mcmapper-test-pg -e POSTGRES_USER=mcmapper -e POSTGRES_PASSWORD=mcmapper -e POSTGRES_DB=mcmapper -p 15432:5432 postgres:17-alpine
|
|
docker run -d --name mcmapper-test-redis -p 16379:6379 redis:7-alpine
|
|
docker run -d --name mcmapper-test-minio -e MINIO_ROOT_USER=mcmapper -e MINIO_ROOT_PASSWORD=mcmapper-dev-only -p 19000:9000 minio/minio:latest server /data
|
|
cd api && DATABASE_URL=postgres://mcmapper:mcmapper@localhost:15432/mcmapper bun run migrate
|
|
```
|
|
|
|
then, from `api/` or `frontend/`:
|
|
|
|
```
|
|
DATABASE_URL=postgres://mcmapper:mcmapper@localhost:15432/mcmapper \
|
|
REDIS_URL=redis://localhost:16379 \
|
|
MINIO_ENDPOINT=localhost MINIO_PORT=19000 MINIO_ACCESS_KEY=mcmapper MINIO_SECRET_KEY=mcmapper-dev-only \
|
|
bun test
|
|
```
|
|
|
|
Each test file creates and tears down its own server row (random token per run) so runs never
|
|
collide with each other or with real dev data — see `api/src/test-helpers.ts`.
|
|
|
|
## Attribution
|
|
|
|
See `THIRD_PARTY_NOTICES.md`.
|