Split web image's large RUN blocks into per-layer steps; add per-template VERSION files to skip unchanged reprovisioning
The registry sits behind a reverse proxy that 413s large blob pushes, so templates/web/Dockerfile's oversized RUN blocks (apt installs, rustup targets, cargo installs) are broken up so no single layer is too big to push. Also adds templates/<name>/VERSION (starting at "1" for all six templates) and has the provision job's push step look up whether that version is already pushed before running coder templates push, since the workflow triggers on any change under templates/** and previously reprovisioned every template on every push, not just the one that changed.
This commit is contained in:
@@ -2,8 +2,13 @@ name: Provision Coder Templates
|
|||||||
|
|
||||||
# Keeps Coder templates in sync with templates/*/ in this repo:
|
# Keeps Coder templates in sync with templates/*/ in this repo:
|
||||||
# - every push to main pushes a new version of each templates/<env>/ dir
|
# - every push to main pushes a new version of each templates/<env>/ dir
|
||||||
# (coder templates push creates it if it doesn't exist yet, so adding a
|
# whose VERSION file names a version not already pushed (coder templates
|
||||||
# new templates/<env>/ directory is enough to provision a new one)
|
# push creates the template if it doesn't exist yet, so adding a new
|
||||||
|
# templates/<env>/ directory - with a VERSION file - is enough to
|
||||||
|
# provision a new one). This workflow triggers on any change under
|
||||||
|
# templates/**, not just a specific template's own directory, so VERSION
|
||||||
|
# is what keeps an edit to one template from generating a no-op new
|
||||||
|
# version for every other, unchanged template.
|
||||||
# - if a templates/<env>/ directory is removed on main, its template is
|
# - if a templates/<env>/ directory is removed on main, its template is
|
||||||
# deleted from Coder. `coder templates delete` refuses to delete a
|
# deleted from Coder. `coder templates delete` refuses to delete a
|
||||||
# template that still has active workspaces, so this can't silently
|
# template that still has active workspaces, so this can't silently
|
||||||
@@ -154,9 +159,34 @@ jobs:
|
|||||||
continue
|
continue
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
|
# templates/<name>/VERSION lets a template opt out of being
|
||||||
|
# reprovisioned on every push: bump it and coder templates push
|
||||||
|
# names the new version "v<N>"; leave it as-is and this looks up
|
||||||
|
# whether that version name is already pushed and skips if so.
|
||||||
|
# This is a manual contract, not a content hash - editing a
|
||||||
|
# template without bumping its VERSION means the change won't
|
||||||
|
# go out until someone does. paths: on this workflow's trigger
|
||||||
|
# is templates/** as a whole, so without this every template
|
||||||
|
# gets a new (identical) version on any push under templates/,
|
||||||
|
# even ones whose own directory didn't change.
|
||||||
|
version=""
|
||||||
|
if [ -f "$dir/VERSION" ]; then
|
||||||
|
version="$(tr -d '[:space:]' < "$dir/VERSION")"
|
||||||
|
fi
|
||||||
|
if [ -n "$version" ]; then
|
||||||
|
existing="$(coder templates versions list "$full" -o json 2>/dev/null || true)"
|
||||||
|
if [ -n "$existing" ] && echo "$existing" | jq -e --arg v "v$version" 'any(.[]; .name == $v)' >/dev/null 2>&1; then
|
||||||
|
echo "Skipping $full - version v$version (templates/$name/VERSION) is already pushed. Bump the VERSION file to push a new one."
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
args=(-d "$dir" --yes -m "auto-provisioned from ${GITHUB_SHA:0:12}")
|
||||||
|
[ -n "$version" ] && args+=(--name "v$version")
|
||||||
|
|
||||||
echo "::group::Pushing $full from $dir"
|
echo "::group::Pushing $full from $dir"
|
||||||
coder templates push "$full" -d "$dir" --yes \
|
coder templates push "$full" "${args[@]}"
|
||||||
-m "auto-provisioned from ${GITHUB_SHA:0:12}"
|
|
||||||
echo "::endgroup::"
|
echo "::endgroup::"
|
||||||
done
|
done
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
1
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
1
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
1
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
1
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
1
|
||||||
+65
-14
@@ -32,27 +32,56 @@ RUN curl -fsSL https://dl.google.com/linux/linux_signing_key.pub \
|
|||||||
|
|
||||||
# Core build toolchain, crypto/DB headers, Tauri 2 / WebKit GUI prerequisites,
|
# Core build toolchain, crypto/DB headers, Tauri 2 / WebKit GUI prerequisites,
|
||||||
# X11 dev libs, DB CLI clients, Python + OpenCV, protobuf compiler.
|
# X11 dev libs, DB CLI clients, Python + OpenCV, protobuf compiler.
|
||||||
|
#
|
||||||
|
# Split into several RUN steps (rather than one big apt-get install) so no
|
||||||
|
# single resulting layer is too large to push to the registry - it sits
|
||||||
|
# behind a reverse proxy with a request body size cap, and a couple of these
|
||||||
|
# packages (llvm, libopencv-dev, libwebkit2gtk-4.1-dev) are individually
|
||||||
|
# large enough to blow past it if lumped together with everything else.
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
build-essential pkg-config cmake \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
clang llvm \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
build-essential pkg-config cmake clang llvm \
|
|
||||||
git git-lfs jq unzip tar file htop tree tmux zsh openssh-client \
|
git git-lfs jq unzip tar file htop tree tmux zsh openssh-client \
|
||||||
|
&& git lfs install --system \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
libssl-dev libpq-dev libsqlite3-dev \
|
libssl-dev libpq-dev libsqlite3-dev \
|
||||||
|
postgresql-client redis-tools sqlite3 \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
libwebkit2gtk-4.1-dev libayatana-appindicator3-dev librsvg2-dev libxdo-dev \
|
libwebkit2gtk-4.1-dev libayatana-appindicator3-dev librsvg2-dev libxdo-dev \
|
||||||
libgtk-3-dev libsoup-3.0-dev \
|
libgtk-3-dev libsoup-3.0-dev \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
libx11-dev libxext-dev libxrender-dev libxtst-dev libxi-dev \
|
libx11-dev libxext-dev libxrender-dev libxtst-dev libxi-dev \
|
||||||
postgresql-client redis-tools sqlite3 \
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
python3 python3-pip python3-venv python3-dev \
|
python3 python3-pip python3-venv python3-dev \
|
||||||
libopencv-dev \
|
|
||||||
protobuf-compiler \
|
protobuf-compiler \
|
||||||
&& git lfs install --system \
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
libopencv-dev \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# Node.js LTS (22.x) plus npm/pnpm/yarn as root so global bins land on the
|
# Node.js LTS (22.x) plus npm/pnpm/yarn as root so global bins land on the
|
||||||
# system PATH for every user.
|
# system PATH for every user.
|
||||||
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
|
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
|
||||||
&& apt-get install -y --no-install-recommends nodejs \
|
&& apt-get install -y --no-install-recommends nodejs \
|
||||||
&& npm install -g pnpm yarn \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN npm install -g pnpm yarn
|
||||||
|
|
||||||
# Global Python prototyping packages: CV, ONNX runtime, CPU-only torch wheel.
|
# Global Python prototyping packages: CV, ONNX runtime, CPU-only torch wheel.
|
||||||
# Ubuntu 24.04's system Python is PEP 668 externally-managed; this is a
|
# Ubuntu 24.04's system Python is PEP 668 externally-managed; this is a
|
||||||
# throwaway container image, so --break-system-packages is the right call
|
# throwaway container image, so --break-system-packages is the right call
|
||||||
@@ -61,9 +90,14 @@ RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
|
|||||||
# file (dpkg-installed, not pip-installed), so `pip install --upgrade pip`
|
# file (dpkg-installed, not pip-installed), so `pip install --upgrade pip`
|
||||||
# fails trying to uninstall it in place - and it's unneeded anyway, the
|
# fails trying to uninstall it in place - and it's unneeded anyway, the
|
||||||
# packages below install fine under the stock version.
|
# packages below install fine under the stock version.
|
||||||
|
#
|
||||||
|
# Each package gets its own RUN/layer for the same reverse-proxy body-size
|
||||||
|
# reason as the apt-get split above - torch's CPU wheel and opencv-python's
|
||||||
|
# wheel are each large enough on their own to be worth isolating.
|
||||||
|
RUN python3 -m pip install --break-system-packages --no-cache-dir numpy
|
||||||
|
RUN python3 -m pip install --break-system-packages --no-cache-dir opencv-python-headless
|
||||||
|
RUN python3 -m pip install --break-system-packages --no-cache-dir onnxruntime
|
||||||
RUN python3 -m pip install --break-system-packages --no-cache-dir \
|
RUN python3 -m pip install --break-system-packages --no-cache-dir \
|
||||||
numpy opencv-python-headless onnxruntime \
|
|
||||||
&& python3 -m pip install --break-system-packages --no-cache-dir \
|
|
||||||
torch --index-url https://download.pytorch.org/whl/cpu
|
torch --index-url https://download.pytorch.org/whl/cpu
|
||||||
|
|
||||||
# Standard non-root dev user with passwordless sudo. Ubuntu 24.04's base
|
# Standard non-root dev user with passwordless sudo. Ubuntu 24.04's base
|
||||||
@@ -88,15 +122,32 @@ WORKDIR /home/coder
|
|||||||
|
|
||||||
# Rust via rustup: stable toolchain, rust-analyzer/clippy/rustfmt/rust-src,
|
# Rust via rustup: stable toolchain, rust-analyzer/clippy/rustfmt/rust-src,
|
||||||
# native + musl targets for x86_64/aarch64, and cargo helper utilities.
|
# native + musl targets for x86_64/aarch64, and cargo helper utilities.
|
||||||
|
#
|
||||||
|
# Split into one RUN per target/tool (rather than one big chained command) so
|
||||||
|
# no single layer is too large to push to the registry, for the same
|
||||||
|
# reverse-proxy body-size reason as the apt-get split above - the base
|
||||||
|
# toolchain and each additional target's std library are each sizeable, and
|
||||||
|
# `cargo install` leaves a build/registry cache behind that needs clearing
|
||||||
|
# inside its own RUN, or it would just bloat that same layer instead.
|
||||||
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y \
|
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y \
|
||||||
--default-toolchain stable --profile default \
|
--default-toolchain stable --profile default \
|
||||||
&& rustup component add rustfmt clippy rust-analyzer rust-src \
|
&& rm -rf "$RUSTUP_HOME"/tmp "$RUSTUP_HOME"/downloads
|
||||||
&& rustup target add \
|
|
||||||
x86_64-unknown-linux-gnu \
|
RUN rustup component add rustfmt clippy rust-analyzer rust-src
|
||||||
x86_64-unknown-linux-musl \
|
|
||||||
aarch64-unknown-linux-gnu \
|
RUN rustup target add x86_64-unknown-linux-gnu
|
||||||
aarch64-unknown-linux-musl \
|
RUN rustup target add x86_64-unknown-linux-musl
|
||||||
&& cargo install --locked cargo-watch cargo-edit cross bacon
|
RUN rustup target add aarch64-unknown-linux-gnu
|
||||||
|
RUN rustup target add aarch64-unknown-linux-musl
|
||||||
|
|
||||||
|
RUN cargo install --locked cargo-watch \
|
||||||
|
&& rm -rf "$CARGO_HOME"/registry "$CARGO_HOME"/git
|
||||||
|
RUN cargo install --locked cargo-edit \
|
||||||
|
&& rm -rf "$CARGO_HOME"/registry "$CARGO_HOME"/git
|
||||||
|
RUN cargo install --locked cross \
|
||||||
|
&& rm -rf "$CARGO_HOME"/registry "$CARGO_HOME"/git
|
||||||
|
RUN cargo install --locked bacon \
|
||||||
|
&& rm -rf "$CARGO_HOME"/registry "$CARGO_HOME"/git
|
||||||
|
|
||||||
# Bun: global runtime for the ElysiaJS backend and fast scripting.
|
# Bun: global runtime for the ElysiaJS backend and fast scripting.
|
||||||
RUN curl -fsSL https://bun.sh/install | bash
|
RUN curl -fsSL https://bun.sh/install | bash
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
1
|
||||||
Reference in New Issue
Block a user