Fixes a real gap left over from Phase 11: mesh.rs's 3D mesher was still
using the hand-picked flat palette instead of texture-averaged colors.
Adds worker/src/atlas.rs to pack downloaded block textures into a single
PNG atlas + UV rect map, threads tile-relative UV and atlas-rect buffers
through the mesh binary format (v2, hard break — meshes are a
regenerable render cache), serves the atlas from MinIO via two new api
routes, and adds a custom Babylon shader that falls back to flat vertex
colors per-fragment for untextured quads. glTF export intentionally
stays vertex-color-only (documented reasoning in gltf-export.js) since
standard glTF materials can't express that same per-fragment fallback.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Layers texture-averaged colors on top of palette.rs's hand-picked table as a
fallback (color_for_textured), not a hard replacement — biome-tinted/animated
blocks (grass top, leaves, water, lava) deliberately keep the hand-picked
color since averaging their raw jar textures would be wrong, not just
imprecise.
Vanilla: worker/src/textures.rs downloads Mojang's official client jar
directly from launchermeta/piston-meta/piston-data (same endpoints the real
launcher uses, gated behind ACCEPT_MINECRAFT_EULA=true, off by default,
mirrors BlueMap's accept-download) and averages every block texture, cached
to disk so it's not re-downloaded every restart.
Modded: ingests block_registry/block_textures messages from the mod (new
Postgres tables + MinIO storage in api/src/textures.ts) — sent once per
connection over the existing WS gateway.
texturepacks/<name>/ (Dynmap-style flat PNGs) lets an operator override the
vanilla defaults worker-wide via TEXTURE_PACK. A per-server texturePack
admin column/API exists for the same purpose, but render-time per-server
resolution (of both the admin selection and the ingested modded textures) is
explicitly deferred — the worker still applies one process-wide palette; true
per-server resolution needs the render pipeline to thread a server-scoped
palette through the batch/GPU-dispatch path, judged too big a change for this
phase. Documented in README.
Docker was unavailable in this dev environment for the usual integration-test
verification; bunx tsc --noEmit was used as a fallback static check instead
(clean except 2 pre-existing unrelated errors in markers.test.ts).
Splits RenderBackend's per-voxel face-visibility extraction and tile
shading out as GPU-offloadable steps (wgpu compute shaders), while
keeping greedy-mesh merge/compaction CPU-only per the plan's "partial
GPU rendering" design. RENDER_BACKEND=cpu|gpu|hybrid selects the
strategy, falling back to cpu automatically (logged) if no compatible
GPU adapter is found. Verified against a real GPU: all tests pass,
including ones asserting byte-identical output between the cpu and
gpu backends; a new benchmark example honestly shows cpu currently
outperforming gpu/hybrid at realistic batch sizes since each call is
its own dispatch/readback round trip rather than batched across a
whole render batch (documented as a follow-up optimization).
Also fixes two real, pre-existing gaps found while validating the
worker's actual `docker build`: a missing .dockerignore was sending
the local multi-GB target/ dir into the build context, and the
Dockerfile's rust:1.80 pin was already too old for current
transitive dependency MSRVs (bumped to rust:1.97).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Relays the mod's throttled player_positions roster over a new
/ws/players/:serverId gateway (Redis pub/sub + snapshot key so a
tab connecting between mod flushes isn't empty), gated per-server
by a new playerPositionsVisible admin toggle independent of the
mod's own tracking config. Frontend renders the roster as map
markers with a show/hide toggle and online count. Covered by unit
tests (players.test.ts, ws-gateway.test.ts, admin.test.ts) and a
new e2e spec that plays the real mod WS protocol from inside a
browser context.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Add worker/src/config.rs resolving RENDER_THREADS (auto | manual override)
and RENDER_PROFILE (server | consumer) into a rayon thread-pool size and a
per-xread batch size, unit tested. Restage main.rs's processing loop into
three stages per batch: async fetch, CPU-bound rasterize+mesh parallelized
across the batch on a sized rayon pool, then async store+ack — the
parallelism target is many chunks in flight at once, since a single 16x16
tile is too small for rayon to help within itself (per the pre-existing
doc comment in render/cpu.rs).
Verified locally: 3 worker instances against the same Redis stream split
24 queued dirty-chunk jobs with zero duplicates and zero drops (confirmed
via worker logs and tile_pointers rows), validating the consumer-group
design ahead of a real remote-worker deployment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Gate a new /api/admin/* route set (register/list/update/delete servers)
behind a single shared MCMAPPER_ADMIN_TOKEN header, and add a /admin
frontend page (Alpine) to unlock, register new servers, and edit
authMode/anonymousChatAllowed/waypointFormat per server — these columns
already existed but were only editable via direct DB edit until now.
Written test-first per the project's TDD workflow: admin.ts's domain
logic, the index.ts route wiring, and a new e2e/tests/admin.spec.ts
covering the token gate and register/edit/delete round trip through the
real browser UI.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
The whole point of driving a real browser instead of curling api/frontend
separately: none of this session's prior "live verification" ever exercised
the same-origin routing production relies on (Caddy: /ws*+/api/* -> api,
else -> frontend), so a real browser's relative fetch()/WebSocket calls were
never actually proven to resolve. e2e/proxy.ts mirrors that routing (no
caddy binary available locally); global-setup.ts/global-teardown.ts
orchestrate throwaway infra + seeded data + the api/frontend/proxy
processes end to end.
Getting the suite green surfaced two genuine bugs invisible to unit tests:
- index.pug loaded map.js via two <script type="module"> tags (one moved to
<head> to fix load-order, the original left in place by mistake), causing
Alpine's x-init="init()" to run twice and Leaflet to throw "Map container
is already initialized" on the second call.
- map.js's exportRegion() passed the Alpine-reactive `regionBounds` object
straight into worker.postMessage(); Alpine wraps assigned state in
Proxies, which the structured clone algorithm can't clone, so every
export silently failed. Fixed by spreading into a plain object first.
Covers the two flows flagged all session as verified only at the unit/curl
level: the marker click-to-place/edit popup (including that a marker
created while linked shows up in a second browser context with the same
session, proving server-side sync) and the region-select drag + glTF
export (including a real triggered file download).
Adds the Region Export API (GET /api/export/:serverId/:dimension/
:x1/:z1/:x2/:z2, chunk-coordinate range, capped at 64 chunks server-
side) streaming raw chunkSections rows for a footprint, plus a fully
client-side pipeline that turns that into a downloadable glTF (.glb):
a JS port of the Rust worker's greedy mesher (voxel-mesh.js, mirrored
test-for-test against worker/src/mesh.rs) and block-color palette
(block-colors.js, mirrored against worker/src/palette.rs), a base64
section decoder, mesh merging with per-section world offsets, and a
hand-rolled minimal GLB writer (gltf-export.js) — hand-rolled rather
than Babylon's GLTF2Export since this project's Babylon usage never
meshes client-side (only renders pre-built buffers) and a live
Scene/Engine isn't needed or unit-testable. The whole mesh+export
pipeline runs inside a Web Worker (export-worker.js) so a multi-chunk
export doesn't block the UI thread. The map UI gets a drag-to-select
region tool (chunk-snapped, live rectangle preview, cap enforced
client-side too) and an Export glTF button.
Built test-first per the project's TDD workflow; the exported .glb
was independently validated with @gltf-transform/cli against real
seeded terrain data (correct triangle count, vertex count, and
bounding box for the selected area).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Markers now carry a caller-supplied x/y/z instead of an auto-resolved
terrain height: clicking the map opens a real Leaflet popup pre-filled
with the clicked x/z, a default y of 60, and a random color, all
editable before saving. The same popup now also opens for editing an
existing marker (new updateMarker/PATCH /api/markers/:markerId, TDD'd
in markers.test.ts/index.test.ts). Removes the now-unused
resolveHeight/GET /api/height machinery. Linked-account markers
already synced server-side via the markers table, so cross-device
sync falls out of the existing loadMarkers()-on-init flow.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Linked accounts can place 2D markers on the map (y auto-derived from the
chunk store's heightmap); anonymous visitors keep a localStorage-only
list via the same height lookup. Sharing a marker forwards a structured
payload to the mod over its WS connection, which builds the actual
chat text (see MCMapper-Mod for the JourneyMap/Xaero formatting).
Built test-first per the project's TDD workflow.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
Test-first from here on (per request after Phase 2): every module below
was written test-then-implementation, confirmed red before green.
api: accounts/sessions/chat_messages tables, plus servers.anonymousChatAllowed.
Online accounts merge into one global identity per real Mojang uuid
(partial unique index on mc_uuid WHERE server_id IS NULL); offline accounts
are scoped per-server (partial unique index on (server_id, mc_uuid)) — see
schema.ts's accounts comment and link.test.ts's merge-scoping tests.
link.ts: storeLinkCode/redeemLinkCode (single-use, Redis-backed with a
10-minute TTL) and session lookup/revocation. Sessions come back in the
HTTP response body rather than an httpOnly cookie — a deliberate MVP
simplification (see link.ts's doc comment) that sidesteps needing to
verify exactly how Elysia's .ws() routes surface cookies; the client
sends the token back via X-MCMapper-Session.
chat.ts/chat-gateway.ts: mod-originated chat (ws-gateway.ts's new "chat"
and "link_request" message types) and browser-originated chat
(/ws/chat/:serverId) both persist to chat_messages and publish to a
per-server Redis pub/sub channel; browser chat additionally resolves
identity (linked session > nickname > rejected if anonymous chat is
disabled for that server) and forwards to the mod's own connection via a
new serverId->socket registry in ws-gateway.ts (getModSocket).
frontend: chat panel + link-code entry on the 2D map page, session token
kept in localStorage (matching the no-cookie tradeoff above).
Verified end-to-end against live containers, including through the real
mod-side Java client: a link code generated by DefaultBackendConnection
round-trips through actual HTTP redemption to the correct account, and a
browser chat message correctly forwards through to the mod's live
ChatListener callback (not just persisted/published).
Requested after Phase 2: from here on, MCMapper development follows
TDD (test-first) — this retrofits the pieces already built before that
request landed.
worker: unit tests for the tile rasterizer (background fill, exact
upscaled-block boundaries, full-grid painting, out-of-bounds columns) and
the block-color palette (distinctness checks, including that the
"unmapped block" placeholder never accidentally collides with a real
block's color). 16 tests total alongside the existing mesher tests.
api: wired up `bun test`. Unit tests for chunkOf's coordinate math.
Integration tests (real Postgres/Redis/MinIO, see README's new "Running
tests" section) for wsGateway.message() — auth accept/reject, upsert +
dedup on columns/sections, not-authenticated/invalid-JSON handling — and
for the tile/mesh/servers HTTP routes, driven through Elysia's in-process
`.handle()` rather than a bound port (sidesteps the stale dev-server
port-collision issue hit repeatedly this session). index.ts now exports
`app` and only calls `.listen()` when run directly, specifically so tests
can drive it this way.
frontend: extracted mesh.js's binary-format parser into its own ESM
module (mesh-format.js) so it's unit-testable without a browser/Babylon;
mesh.js now imports it. Tests build a buffer independently of the parser
(mirroring worker's encoder layout) so a mismatch in either direction —
Rust producer or JS consumer drifting — would be caught.
api: chunk_sections table (per 16x16x16 section, base64-encoded u16
blockStateId array) and mesh_pointers table, additive to Phase 1's
column-based chunk_columns/tile_pointers — 2D tile rendering keeps using
the cheap column path unchanged. New "sections" WS message (backfill on
chunk load + delta resend on flush, same "current state, not a diff"
philosophy as columns) reuses the existing dirty-chunk Redis event, so one
event now triggers the worker to re-render both the 2D tile and any 3D
meshes for that chunk. New mesh-serving routes.
worker: a from-scratch greedy mesher (per-axis 2D mask sweep + rectangle
merge — the standard voxel-meshing technique, reimplemented from its
public description, not copied from any codebase) producing a compact
custom binary vertex buffer per non-empty section. Verified with unit
tests, including one that specifically checks a uniform section collapses
to exactly 6 merged quads rather than one quad per voxel face (the
decisive signal that merging, not just per-voxel face emission, is
actually happening).
frontend: a barebones Babylon.js 3D viewer (/3d) that loads a fixed radius
of chunks, parses the mesh binary format, and renders each section as its
own mesh (no cross-section merging yet, no camera-based streaming yet —
both reasonable follow-ups once there's a reason to optimize).
End-to-end verified against live containers, including through the real
mod-side Java WS client (see MCMapper-Mod's matching commit): a known
half-solid section correctly round-trips to exactly 24 vertices / 36
indices at the mesh-serving endpoint, matching the "6 merged outer faces"
the unit tests predict.
api: WS gateway with token auth (Postgres-backed servers table, seeded via
`bun run seed`), column-granularity chunk store (hand-written SQL
migrations, no drizzle-kit CLI — its config loader needs esbuild, which
doesn't install cleanly here), dirty-chunk Redis stream producer with
per-flush dedup, and a tile-serving route.
worker: consumes the dirty-chunk stream via a proper consumer group,
rasterizes each chunk's columns into a single-resolution top-down PNG
(static pre-Flattening block-id palette), uploads to MinIO, and upserts
the tile pointer.
frontend: barebones Leaflet 2D viewer (CRS.Simple, one native zoom level)
wired to /api/servers and /api/tiles.
Object storage: tiles live in a dedicated `mcmapper-tiles` bucket on the
existing shared MinIO instance (devstack-minio on octo-winsrv) instead of
a per-stack container, via a scoped access key limited to that one bucket
— see README's "Object storage" section. MINIO_SECRET_KEY is real and is
deliberately not committed; docker-compose layers an untracked .env over
.env.example for it.
Full pipeline verified end-to-end against live containers: WS auth ->
Postgres upsert -> deduped Redis dirty-chunk event -> worker rasterize ->
MinIO upload -> tile fetch through the api route, including from the
actual mod-side WS client (see MCMapper-Mod's matching commit).
api/ (ElysiaJS+Bun WS gateway skeleton), worker/ (Rust, Redis dirty-chunk
stream consumer behind a swappable RenderBackend trait), frontend/
(ElysiaJS+Pug+Tailwind4+Alpine, one server-rendered page) — all three
verified running locally. docker-compose wires them up with postgres,
redis, minio (rendered tile/mesh object storage), and Caddy as reverse
proxy.
octoturge