s3.octoturge.com normalizes an explicit :443/:80 out of the Host header
before validating SigV4, so including it caused SignatureDoesNotMatch
even with correct credentials.
rust:1.97 (unsuffixed) now resolves to a trixie base, whose newer glibc
produced a binary the bookworm-slim runtime stage couldn't load
("GLIBC_2.38 not found (required by mcmapper-worker)"), crash-looping the
container on every start. Pinning the builder to rust:1.97-bookworm matches
the runtime's Debian release. Confirmed by rebuilding and running the real
image — worker now starts and runs (its next failure is an unrelated,
expected MinIO connectivity gap, not this).
Fixed Windows-to-WSL2 Docker connectivity (real dockerd runs inside
the WSL Ubuntu distro, not Docker Desktop) so the Phase 13 test that
had been written but unrun could actually be exercised. api (98/98)
and frontend (39/39) suites both pass against real Postgres/Redis/
MinIO now.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Adds worker/src/models.rs (parent-chain blockstate/model resolution,
texture-variable substitution reusing Phase 12's atlas keys), routes
non-cube blocks through new per-element mesh emission in mesh.rs while
leaving full-cube blocks on the existing cube mesher, and threads a
per-server ModelContext (vanilla worker-wide + modded per-job) through
main.rs. Modded model JSON is stored in a new block_models Postgres
table and read alongside the existing (previously write-only)
block_registry table. Fixes a pre-existing face-culling bug as a side
effect of excluding non-cube voxels from the cube mesher's input.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Fixes a real gap left over from Phase 11: mesh.rs's 3D mesher was still
using the hand-picked flat palette instead of texture-averaged colors.
Adds worker/src/atlas.rs to pack downloaded block textures into a single
PNG atlas + UV rect map, threads tile-relative UV and atlas-rect buffers
through the mesh binary format (v2, hard break — meshes are a
regenerable render cache), serves the atlas from MinIO via two new api
routes, and adds a custom Babylon shader that falls back to flat vertex
colors per-fragment for untextured quads. glTF export intentionally
stays vertex-color-only (documented reasoning in gltf-export.js) since
standard glTF materials can't express that same per-fragment fallback.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Layers texture-averaged colors on top of palette.rs's hand-picked table as a
fallback (color_for_textured), not a hard replacement — biome-tinted/animated
blocks (grass top, leaves, water, lava) deliberately keep the hand-picked
color since averaging their raw jar textures would be wrong, not just
imprecise.
Vanilla: worker/src/textures.rs downloads Mojang's official client jar
directly from launchermeta/piston-meta/piston-data (same endpoints the real
launcher uses, gated behind ACCEPT_MINECRAFT_EULA=true, off by default,
mirrors BlueMap's accept-download) and averages every block texture, cached
to disk so it's not re-downloaded every restart.
Modded: ingests block_registry/block_textures messages from the mod (new
Postgres tables + MinIO storage in api/src/textures.ts) — sent once per
connection over the existing WS gateway.
texturepacks/<name>/ (Dynmap-style flat PNGs) lets an operator override the
vanilla defaults worker-wide via TEXTURE_PACK. A per-server texturePack
admin column/API exists for the same purpose, but render-time per-server
resolution (of both the admin selection and the ingested modded textures) is
explicitly deferred — the worker still applies one process-wide palette; true
per-server resolution needs the render pipeline to thread a server-scoped
palette through the batch/GPU-dispatch path, judged too big a change for this
phase. Documented in README.
Docker was unavailable in this dev environment for the usual integration-test
verification; bunx tsc --noEmit was used as a fallback static check instead
(clean except 2 pre-existing unrelated errors in markers.test.ts).
Splits RenderBackend's per-voxel face-visibility extraction and tile
shading out as GPU-offloadable steps (wgpu compute shaders), while
keeping greedy-mesh merge/compaction CPU-only per the plan's "partial
GPU rendering" design. RENDER_BACKEND=cpu|gpu|hybrid selects the
strategy, falling back to cpu automatically (logged) if no compatible
GPU adapter is found. Verified against a real GPU: all tests pass,
including ones asserting byte-identical output between the cpu and
gpu backends; a new benchmark example honestly shows cpu currently
outperforming gpu/hybrid at realistic batch sizes since each call is
its own dispatch/readback round trip rather than batched across a
whole render batch (documented as a follow-up optimization).
Also fixes two real, pre-existing gaps found while validating the
worker's actual `docker build`: a missing .dockerignore was sending
the local multi-GB target/ dir into the build context, and the
Dockerfile's rust:1.80 pin was already too old for current
transitive dependency MSRVs (bumped to rust:1.97).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Relays the mod's throttled player_positions roster over a new
/ws/players/:serverId gateway (Redis pub/sub + snapshot key so a
tab connecting between mod flushes isn't empty), gated per-server
by a new playerPositionsVisible admin toggle independent of the
mod's own tracking config. Frontend renders the roster as map
markers with a show/hide toggle and online count. Covered by unit
tests (players.test.ts, ws-gateway.test.ts, admin.test.ts) and a
new e2e spec that plays the real mod WS protocol from inside a
browser context.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Add worker/src/config.rs resolving RENDER_THREADS (auto | manual override)
and RENDER_PROFILE (server | consumer) into a rayon thread-pool size and a
per-xread batch size, unit tested. Restage main.rs's processing loop into
three stages per batch: async fetch, CPU-bound rasterize+mesh parallelized
across the batch on a sized rayon pool, then async store+ack — the
parallelism target is many chunks in flight at once, since a single 16x16
tile is too small for rayon to help within itself (per the pre-existing
doc comment in render/cpu.rs).
Verified locally: 3 worker instances against the same Redis stream split
24 queued dirty-chunk jobs with zero duplicates and zero drops (confirmed
via worker logs and tile_pointers rows), validating the consumer-group
design ahead of a real remote-worker deployment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Gate a new /api/admin/* route set (register/list/update/delete servers)
behind a single shared MCMAPPER_ADMIN_TOKEN header, and add a /admin
frontend page (Alpine) to unlock, register new servers, and edit
authMode/anonymousChatAllowed/waypointFormat per server — these columns
already existed but were only editable via direct DB edit until now.
Written test-first per the project's TDD workflow: admin.ts's domain
logic, the index.ts route wiring, and a new e2e/tests/admin.spec.ts
covering the token gate and register/edit/delete round trip through the
real browser UI.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
The whole point of driving a real browser instead of curling api/frontend
separately: none of this session's prior "live verification" ever exercised
the same-origin routing production relies on (Caddy: /ws*+/api/* -> api,
else -> frontend), so a real browser's relative fetch()/WebSocket calls were
never actually proven to resolve. e2e/proxy.ts mirrors that routing (no
caddy binary available locally); global-setup.ts/global-teardown.ts
orchestrate throwaway infra + seeded data + the api/frontend/proxy
processes end to end.
Getting the suite green surfaced two genuine bugs invisible to unit tests:
- index.pug loaded map.js via two <script type="module"> tags (one moved to
<head> to fix load-order, the original left in place by mistake), causing
Alpine's x-init="init()" to run twice and Leaflet to throw "Map container
is already initialized" on the second call.
- map.js's exportRegion() passed the Alpine-reactive `regionBounds` object
straight into worker.postMessage(); Alpine wraps assigned state in
Proxies, which the structured clone algorithm can't clone, so every
export silently failed. Fixed by spreading into a plain object first.
Covers the two flows flagged all session as verified only at the unit/curl
level: the marker click-to-place/edit popup (including that a marker
created while linked shows up in a second browser context with the same
session, proving server-side sync) and the region-select drag + glTF
export (including a real triggered file download).
Adds the Region Export API (GET /api/export/:serverId/:dimension/
:x1/:z1/:x2/:z2, chunk-coordinate range, capped at 64 chunks server-
side) streaming raw chunkSections rows for a footprint, plus a fully
client-side pipeline that turns that into a downloadable glTF (.glb):
a JS port of the Rust worker's greedy mesher (voxel-mesh.js, mirrored
test-for-test against worker/src/mesh.rs) and block-color palette
(block-colors.js, mirrored against worker/src/palette.rs), a base64
section decoder, mesh merging with per-section world offsets, and a
hand-rolled minimal GLB writer (gltf-export.js) — hand-rolled rather
than Babylon's GLTF2Export since this project's Babylon usage never
meshes client-side (only renders pre-built buffers) and a live
Scene/Engine isn't needed or unit-testable. The whole mesh+export
pipeline runs inside a Web Worker (export-worker.js) so a multi-chunk
export doesn't block the UI thread. The map UI gets a drag-to-select
region tool (chunk-snapped, live rectangle preview, cap enforced
client-side too) and an Export glTF button.
Built test-first per the project's TDD workflow; the exported .glb
was independently validated with @gltf-transform/cli against real
seeded terrain data (correct triangle count, vertex count, and
bounding box for the selected area).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Markers now carry a caller-supplied x/y/z instead of an auto-resolved
terrain height: clicking the map opens a real Leaflet popup pre-filled
with the clicked x/z, a default y of 60, and a random color, all
editable before saving. The same popup now also opens for editing an
existing marker (new updateMarker/PATCH /api/markers/:markerId, TDD'd
in markers.test.ts/index.test.ts). Removes the now-unused
resolveHeight/GET /api/height machinery. Linked-account markers
already synced server-side via the markers table, so cross-device
sync falls out of the existing loadMarkers()-on-init flow.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Linked accounts can place 2D markers on the map (y auto-derived from the
chunk store's heightmap); anonymous visitors keep a localStorage-only
list via the same height lookup. Sharing a marker forwards a structured
payload to the mod over its WS connection, which builds the actual
chat text (see MCMapper-Mod for the JourneyMap/Xaero formatting).
Built test-first per the project's TDD workflow.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Test-first from here on (per request after Phase 2): every module below
was written test-then-implementation, confirmed red before green.
api: accounts/sessions/chat_messages tables, plus servers.anonymousChatAllowed.
Online accounts merge into one global identity per real Mojang uuid
(partial unique index on mc_uuid WHERE server_id IS NULL); offline accounts
are scoped per-server (partial unique index on (server_id, mc_uuid)) — see
schema.ts's accounts comment and link.test.ts's merge-scoping tests.
link.ts: storeLinkCode/redeemLinkCode (single-use, Redis-backed with a
10-minute TTL) and session lookup/revocation. Sessions come back in the
HTTP response body rather than an httpOnly cookie — a deliberate MVP
simplification (see link.ts's doc comment) that sidesteps needing to
verify exactly how Elysia's .ws() routes surface cookies; the client
sends the token back via X-MCMapper-Session.
chat.ts/chat-gateway.ts: mod-originated chat (ws-gateway.ts's new "chat"
and "link_request" message types) and browser-originated chat
(/ws/chat/:serverId) both persist to chat_messages and publish to a
per-server Redis pub/sub channel; browser chat additionally resolves
identity (linked session > nickname > rejected if anonymous chat is
disabled for that server) and forwards to the mod's own connection via a
new serverId->socket registry in ws-gateway.ts (getModSocket).
frontend: chat panel + link-code entry on the 2D map page, session token
kept in localStorage (matching the no-cookie tradeoff above).
Verified end-to-end against live containers, including through the real
mod-side Java client: a link code generated by DefaultBackendConnection
round-trips through actual HTTP redemption to the correct account, and a
browser chat message correctly forwards through to the mod's live
ChatListener callback (not just persisted/published).
Requested after Phase 2: from here on, MCMapper development follows
TDD (test-first) — this retrofits the pieces already built before that
request landed.
worker: unit tests for the tile rasterizer (background fill, exact
upscaled-block boundaries, full-grid painting, out-of-bounds columns) and
the block-color palette (distinctness checks, including that the
"unmapped block" placeholder never accidentally collides with a real
block's color). 16 tests total alongside the existing mesher tests.
api: wired up `bun test`. Unit tests for chunkOf's coordinate math.
Integration tests (real Postgres/Redis/MinIO, see README's new "Running
tests" section) for wsGateway.message() — auth accept/reject, upsert +
dedup on columns/sections, not-authenticated/invalid-JSON handling — and
for the tile/mesh/servers HTTP routes, driven through Elysia's in-process
`.handle()` rather than a bound port (sidesteps the stale dev-server
port-collision issue hit repeatedly this session). index.ts now exports
`app` and only calls `.listen()` when run directly, specifically so tests
can drive it this way.
frontend: extracted mesh.js's binary-format parser into its own ESM
module (mesh-format.js) so it's unit-testable without a browser/Babylon;
mesh.js now imports it. Tests build a buffer independently of the parser
(mirroring worker's encoder layout) so a mismatch in either direction —
Rust producer or JS consumer drifting — would be caught.
api: chunk_sections table (per 16x16x16 section, base64-encoded u16
blockStateId array) and mesh_pointers table, additive to Phase 1's
column-based chunk_columns/tile_pointers — 2D tile rendering keeps using
the cheap column path unchanged. New "sections" WS message (backfill on
chunk load + delta resend on flush, same "current state, not a diff"
philosophy as columns) reuses the existing dirty-chunk Redis event, so one
event now triggers the worker to re-render both the 2D tile and any 3D
meshes for that chunk. New mesh-serving routes.
worker: a from-scratch greedy mesher (per-axis 2D mask sweep + rectangle
merge — the standard voxel-meshing technique, reimplemented from its
public description, not copied from any codebase) producing a compact
custom binary vertex buffer per non-empty section. Verified with unit
tests, including one that specifically checks a uniform section collapses
to exactly 6 merged quads rather than one quad per voxel face (the
decisive signal that merging, not just per-voxel face emission, is
actually happening).
frontend: a barebones Babylon.js 3D viewer (/3d) that loads a fixed radius
of chunks, parses the mesh binary format, and renders each section as its
own mesh (no cross-section merging yet, no camera-based streaming yet —
both reasonable follow-ups once there's a reason to optimize).
End-to-end verified against live containers, including through the real
mod-side Java WS client (see MCMapper-Mod's matching commit): a known
half-solid section correctly round-trips to exactly 24 vertices / 36
indices at the mesh-serving endpoint, matching the "6 merged outer faces"
the unit tests predict.
api: WS gateway with token auth (Postgres-backed servers table, seeded via
`bun run seed`), column-granularity chunk store (hand-written SQL
migrations, no drizzle-kit CLI — its config loader needs esbuild, which
doesn't install cleanly here), dirty-chunk Redis stream producer with
per-flush dedup, and a tile-serving route.
worker: consumes the dirty-chunk stream via a proper consumer group,
rasterizes each chunk's columns into a single-resolution top-down PNG
(static pre-Flattening block-id palette), uploads to MinIO, and upserts
the tile pointer.
frontend: barebones Leaflet 2D viewer (CRS.Simple, one native zoom level)
wired to /api/servers and /api/tiles.
Object storage: tiles live in a dedicated `mcmapper-tiles` bucket on the
existing shared MinIO instance (devstack-minio on octo-winsrv) instead of
a per-stack container, via a scoped access key limited to that one bucket
— see README's "Object storage" section. MINIO_SECRET_KEY is real and is
deliberately not committed; docker-compose layers an untracked .env over
.env.example for it.
Full pipeline verified end-to-end against live containers: WS auth ->
Postgres upsert -> deduped Redis dirty-chunk event -> worker rasterize ->
MinIO upload -> tile fetch through the api route, including from the
actual mod-side WS client (see MCMapper-Mod's matching commit).
api/ (ElysiaJS+Bun WS gateway skeleton), worker/ (Rust, Redis dirty-chunk
stream consumer behind a swappable RenderBackend trait), frontend/
(ElysiaJS+Pug+Tailwind4+Alpine, one server-rendered page) — all three
verified running locally. docker-compose wires them up with postgres,
redis, minio (rendered tile/mesh object storage), and Caddy as reverse
proxy.
octoturge