Phase 7 (partial): CPU thread/profile config + verified multi-worker scaling
Add worker/src/config.rs resolving RENDER_THREADS (auto | manual override) and RENDER_PROFILE (server | consumer) into a rayon thread-pool size and a per-xread batch size, unit tested. Restage main.rs's processing loop into three stages per batch: async fetch, CPU-bound rasterize+mesh parallelized across the batch on a sized rayon pool, then async store+ack — the parallelism target is many chunks in flight at once, since a single 16x16 tile is too small for rayon to help within itself (per the pre-existing doc comment in render/cpu.rs). Verified locally: 3 worker instances against the same Redis stream split 24 queued dirty-chunk jobs with zero duplicates and zero drops (confirmed via worker logs and tile_pointers rows), validating the consumer-group design ahead of a real remote-worker deployment. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015tKdPZt78zbPUZMXWzKEKt
This commit is contained in:
@@ -11,3 +11,17 @@ MINIO_SECRET_KEY=changeme-set-in-untracked-env
|
||||
|
||||
# cpu | gpu | hybrid — see render::backend. Only `cpu` exists so far (Phase 8 adds gpu/hybrid).
|
||||
RENDER_BACKEND=cpu
|
||||
|
||||
# Thread count for the rayon pool that renders a batch of dirty chunks in parallel (see
|
||||
# config.rs/main.rs — a single 16x16 tile is too small for rayon to help within itself, so the
|
||||
# parallelism target is many chunks in flight at once). `auto` (default) uses
|
||||
# std::thread::available_parallelism(); set a positive integer to override (0/garbage also falls
|
||||
# back to auto).
|
||||
RENDER_THREADS=auto
|
||||
|
||||
# server | consumer — tunes how many dirty-chunk entries are pulled off the Redis stream per
|
||||
# xread cycle before being rendered in parallel. `server` (default) batches more aggressively —
|
||||
# many small batches so every core on a many-core Xeon-style box stays fed; `consumer` uses
|
||||
# fewer, larger batches, less scheduling overhead per chunk on fewer/faster cores. Unrecognized
|
||||
# values fall back to `server`, matching this project's own primary deployment target.
|
||||
RENDER_PROFILE=server
|
||||
|
||||
Reference in New Issue
Block a user