release build tiers¶

Field	Value
Date	2026-07-10
Status	Accepted
Deciders	Holger Zahnleiter
Supersedes	—
Superseded by	—

Context and Problem Statement¶

Generated projects (C++, Go, Java; libraries, services, CLI tools) need a CI/CD pipeline design that is defined once at the architecture level and then implemented per language/CI provider in the generator templates. The design must cover:

which triggers run which work,
how build artifacts flow between stages,
how Debug and Release builds are tested and shipped, and
how a released artifact is packaged, published, and — if necessary — debugged in production.

Without an explicit decision, each language/provider template would invent its own trigger rules, artifact retention, and Debug/Release handling, causing drift between generated projects and making ci_cd.md impossible to enforce consistently.

Scope note: this ADR governs the pipeline generated into new projects. This repository's own .gitlab-ci.yml builds and tests the generator itself (a single Go binary with no Debug/Release artifact split to speak of) and is intentionally not subject to this tiering model — see scripts/selfcheck.sh and make claude-md-check for the self-hosting checks this repository's own pipeline runs instead.

Decision Drivers¶

Fast feedback on every push (Pre-Check within 1-2 minutes).
An artifact is compiled once and reused unchanged through test, package, and publish steps (consistency between what is tested and what ships; avoids redundant compute).
Full release confidence (Debug build with sanitizers and Release build validation) is required only where it matters — at tag time — without slowing down everyday development iteration.
This is a single-maintainer project: process overhead (e.g. release-candidate tags, manual approval gates) should only be introduced when a concrete need exists (YAGNI), not speculatively.
Uniform retention policy across build artifacts, documentation artifacts, and quality/test reports, to bound storage growth while keeping released artifacts indefinitely.
Security and quality gates (SAST, dependency/image scanning, coverage threshold) must be enforced without disproportionately slowing the fast feedback loop.
The design must generalize across C++, Go, and Java. CI-provider-specific mechanics (e.g. the exact GitLab Pages job name, GitHub Actions equivalents) are deferred to per-language/ provider templates, not hardcoded into this ADR.
Production troubleshooting must not require shipping a differently-compiled artifact by default — recompiled/instrumented builds change timing and behavior (Heisenbug risk) and must remain an explicit, temporary escalation, not the default incident-response path.
main must never carry a commit that was not already fully validated at the exact tree state it lands as — enforced by branch protection (merge requires the branch to be up to date with main; no direct pushes), not by re-running tests again after the fact.
Promoting a release should reuse an artifact that was already built and tested, by re-tagging it in the registry, rather than recompiling it — extending the build-once principle across pipeline runs, not just within a single one.
CHANGELOG.md is generated exclusively by git-cliff at tag time (see git.md) and must be available before documentation is published or a release is created.

Considered Options¶

Single flat pipeline — every job runs on every push, no tiering by trigger or build type.
GitFlow-style pipeline with release-candidate tags (vX.Y.Z-RCn) and a manual promotion/approval step before the final release tag.
Trigger-tiered pipeline (Pre-Check → Build → Test → Package & Publish/Deploy) with an MR tier (Debug-only, fast), a Merge-to-main tier (Debug + Release, full validation), and a tag tier that only promotes/publishes — no release-candidate tags, a build-once principle extended across pipeline runs, and a unified retention policy. (chosen)

Decision Outcome¶

Chosen option: Option 3. It gives fast, cheap feedback on the common path (feature branches, MRs) while concentrating the expensive, high-confidence checks (Release-build testing) at merge to main — the point where code becomes part of the shippable history. A version tag then does not test or build anything new; it promotes the already-validated Merge-to-main artifact and publishes it. Release-candidate tags (Option 2) are deliberately not adopted; see the dedicated note under "Trigger Strategy" and the "Pros and Cons" section below.

Trigger Strategy¶

Push to feature branch (no MR): runs the Pre-Check stage only.
Merge Request (open / updated): runs the full development pipeline (MR tier).
Merge to main: runs the Merge-to-main tier — builds and Release-tests the artifact that a later tag will promote.
Tag (vX.Y.Z) on main: runs the promotion pipeline — no rebuild, no re-test; see "Build Configurations".

Precondition: a Merge Request may only be merged once its branch is up to date with main (rebase/merge required) and its pipeline is green. Direct pushes to main are disabled. This guarantees that the commit landing on main is exactly the tree the MR pipeline already validated — nothing new needs to be re-tested purely because a merge happened.

Note — release candidates (vX.Y.Z-RCn): Deliberately not used. Release-candidate tags earn their cost when a release must be signed off by someone other than the author before the immutable final tag is cut, or when the release step itself is risky/hard to reverse (git.md forbids moving or deleting released tags). Here, the automatic intermediate build on every merge to main (latest/nightly, see "Package & Publish / Deploy" below) is already Release-tested, and tagging itself only promotes/re-tags that artifact rather than building or testing anything new (see "Build Configurations") — the release step carries very little of its own risk, further reducing the case for a separate RC step. Revisit this decision once external contributors join, or once a deploy step becomes hard to reverse (e.g. flashing physical hardware).

Global Behavior¶

Interruptible: jobs on feature branches are automatically cancelled when a new push lands on the same branch.
Caching: dependency directories (e.g. package manager caches) and build caches (ccache) are cached per branch, falling back to the main cache.
Build-once principle: every artifact (binary, firmware image, documentation) is built exactly once per pipeline run. All downstream stages (Test, Package, Publish) reuse the artifacts produced in the Build stage — never recompiled, possibly with different compiler options. This protects both pipeline performance and consistency between the tested and the shipped artifact. For the software artifact, this extends across pipeline runs: a tag promotes the Merge-to-main run's artifact instead of rebuilding it (see "Build Configurations").
Retention policy — applies uniformly to build artifacts, documentation artifacts, and quality/test reports:
MR/feature-branch pipelines: 7 days.
Merge-to-main intermediate artifacts (image, docs): retained until superseded by the next merge's intermediate artifact or promoted by a tag, whichever comes first — not bound to the 7-day window, since a later tag needs to promote exactly this artifact (see "Build Configurations").
Release pipelines (Git tag): retained indefinitely.

Build Configurations (Debug vs. Release)¶

Debug: assertions and sanitizers enabled, no optimization. Used for development and testing only.
Release: optimized, no debug symbols embedded, no sanitizers. The only configuration that is ever shipped.

MR tier (feature branch, Merge Request): Debug build only. Both Unit Tests and Integration/Dynamic Tests run exclusively against Debug — fast, everyday feedback.

Merge-to-main tier: because merging requires the branch to be up to date with main and green (see "Trigger Strategy"), the commit landing on main is exactly what the MR tier already validated against Debug — repeating that here would be redundant. This tier instead adds exactly what the MR tier does not cover: it builds the Release configuration and runs Integration/Dynamic Tests against it, catching optimization-related regressions before a release is ever cut. The result is packaged as the intermediate image, tagged with the commit SHA — this is the one artifact a later version tag can promote (see below).

Tag / promotion tier (Git tag): no recompiling, no re-testing. Tagging looks up the intermediate image already built and Release-tested for that exact commit (Merge-to-main tier) and re-tags it as vX.Y.Z in the registry — a registry operation, not a rebuild, extending the build-once principle across pipeline runs rather than only within one. A fresh Security Scan still runs against the (unchanged) image, since vulnerability databases evolve independently of image contents. If no valid intermediate image exists for this commit (e.g. retention lapsed), the pipeline falls back to running the Merge-to-main tier once before promoting — a safety net, not the common path.

Production debugging (escalation path):

Observability first (structured logs, traces, metrics — see monitoring.md).
If needed: extend access/tooling on the same Release artifact (e.g. remote debugger attach, profiler, shell access to an otherwise identical container) — runtime behavior is unchanged. The concrete mechanism (e.g. JDWP/JFR for Java, pprof/Delve for Go, split debug symbols + gdb for C++) is defined in the respective language templates, not in this ADR.
Only as a last resort, time-boxed and on a single instance: manually deploy a differently compiled artifact (e.g. with sanitizers or without optimization). Never as a default rollout — debug flags change timing/behavior and can mask or shift the bug being chased (Heisenbug risk), in addition to the performance overhead of sanitizers under real load.

Debug symbols/metadata for the Release build are retained as a separate artifact with release-level retention where feasible, so stack traces and crash reports can be symbolicated after the fact without deploying a different build. A separate, manually triggered "debug-capable" image/package (same artifact plus extended debugging tooling/access) may be built alongside the release image; it is never deployed automatically.

Pipeline Stages¶

Stage 1: Pre-Check (Linting & Static Analysis)¶

Fast feedback within 1-2 minutes. Runs on every push.

Job: Format & Lint: checks code formatting.
Job: Static Analysis: SAST scans, static compiler checks, linters.
Job: Hardware DRC: automated design-rule checks (e.g. for PCB layouts), if relevant files changed.

Stage 2: Build¶

Runs on MRs, merges to main, and tags. Produces artifacts.

Job: Generate Changelog: runs git-cliff to regenerate CHANGELOG.md from Conventional Commits since the last tag (see git.md); commits the update to main as a normal follow-up commit — the tag itself is never moved. Runs first, before "Build Docs" below, which depends on its output.
Trigger: Git tags only.
Job: Build Software: compiles the code, using compiler caches to speed things up. Debug build on the MR tier; Release build on the Merge-to-main tier. Not rebuilt on tags — a tag promotes the Merge-to-main tier's already-tested Release build instead (see "Build Configurations" above).
Artifacts: executables, firmware binaries (retention: see "Global Behavior").
Job: Build Docs: generates micro-sites or PDF documentation, including a page rendered from CHANGELOG.md. Rebuilt fresh on the tag pipeline too — unlike the software artifact, docs are not promoted unchanged, since they must embed the CHANGELOG.md that "Generate Changelog" only just produced.
Artifacts: HTML folder (retention: see "Global Behavior").

Stage 3: Test¶

Consumes the Stage 2 artifacts. Runs in parallel.

Job: Unit Tests: runs isolated tests, generates JUnit XML and coverage reports. Always runs against the Debug build (see "Build Configurations").
Artifacts: test reports (retention: see "Global Behavior").
Coverage gate: default threshold 80% line coverage; the pipeline fails below it. Configurable/disableable per project (e.g. for heavily hardware-dependent embedded code with a structurally limited achievable coverage).
Job: Integration/Dynamic Tests: heavier tests (dry runs, possibly hardware-in-the-loop simulations). Runs against Debug on non-draft MRs, and against Release on merge to main (see "Build Configurations"). Not re-run on tags — tagging promotes the Merge-to-main tier's already-tested Release build instead.

Stage 4: Package & Publish / Deploy¶

Never recompiles. Reuses artifacts built in Stage 2 of this pipeline run (Docs, Changelog), or — for the Release image on tags — the artifact built and Release-tested in a previous Merge-to-main run (build-once principle, see "Global Behavior" and "Build Configurations"). Triggers differ per job.

Job: Push Intermediate Image: packages the build artifacts into a container image or package (tag: vX.Y.Z-commit), keyed by commit SHA so a later tag can find and promote it.
Trigger: manual only on MRs. Automatic on merge to main (as latest/nightly).
Job: Security Scan: scans the built image/package for known vulnerabilities (e.g. Trivy). Fails on High/Critical findings.
Trigger: after every image/package build (intermediate). Also re-run at promotion time (tag) even though the image itself does not change — vulnerability databases evolve independently of image contents.
Job: Promote Release Image: re-tags the intermediate image already built and Release-tested for this exact commit (Merge-to-main tier) as vX.Y.Z in the registry — a registry operation, not a rebuild. Falls back to running the Merge-to-main tier once first if no valid intermediate image exists for this commit (e.g. retention lapsed, see "Global Behavior").
Trigger: Git tags (vX.Y.Z) only.
Job: Push Debug-Capable Image (optional): packages the same Release build with extended debugging access (e.g. shell, remote debugger port) for the escalation path described under "Build Configurations" — no separate Debug build, no recompiling.
Trigger: manual only. Never deployed automatically.
Job: Publish Docs: takes the HTML artifacts freshly built in Stage 2 (including the CHANGELOG.md page, see Stage 2 "Job: Generate Changelog") and publishes them. The concrete target platform/mechanism (e.g. GitLab Pages, GitHub Pages) is CI-provider-specific and is defined in the generator templates, not in this ADR.
Trigger: Git tags on main only.
Job: Create Release: creates an automated release in the repository, using the CHANGELOG.md entry produced by Stage 2 "Job: Generate Changelog" as release notes, and attaches the final binaries as assets.
Trigger: Git tags on main only.

Positive Consequences¶

Feature/MR pipelines stay fast: only a Debug build and Debug-based tests run by default.
main cannot be "polluted" by an unvalidated commit: branch protection guarantees every commit on main was tested at its exact tree state, and the Merge-to-main tier adds Release-level validation before anything is tagged.
Tagging is a low-risk promotion, not a new build: the artifact that is release-tested on merge to main is the exact artifact re-tagged as vX.Y.Z — no drift from recompilation, and very little can go wrong in the tag pipeline itself.
No release-candidate process overhead for a single-maintainer or small-team project, while the Merge-to-main tier's intermediate build still provides a fully Release-tested dry run of packaging.
A single retention policy is easy to reason about and enforce across artifact types.
Production debugging has a defined, low-risk escalation path that defaults to reusing the shipped Release artifact rather than redeploying a different build.

Negative Consequences / Risks¶

Re-tagging an image without rebuilding requires registry tooling that supports promotion by reference (e.g. crane/skopeo, or the CI provider's native retag mechanism) rather than a plain docker build/push — an extra piece of tooling to standardize across languages.
Intermediate artifacts must be retained until they are either superseded or promoted, not just for a fixed 7-day window — slightly more retention logic than a flat policy.
git-cliff regenerating and committing CHANGELOG.md after the tag requires CI to have write access to the otherwise-protected main branch for that one automated commit — a least-privilege service credential scoped to this single operation (see security.md).
The optional "debug-capable" image adds an extra manually triggered job per language template to implement and keep in sync with the release image build.
If the Merge-to-main tier's Release build/Integration Tests fail, that failure surfaces on main (post-merge) rather than blocking the merge itself — main may briefly show a red intermediate-build pipeline until fixed. Accepted trade-off to keep MR feedback fast; the broken state is visible immediately and blocks any tag/promotion until resolved.

Pros and Cons of the Options¶

Option 1 — Single flat pipeline¶

Pro: simplest to implement; no tiering logic.
Con: every push pays the cost of the full test/build matrix, slowing down everyday iteration.
Con: no distinction between a "preview" artifact and a release-validated one.

Option 2 — GitFlow-style pipeline with release-candidate tags¶

Pro: allows a release to be validated and signed off before the immutable final tag.
Pro: well suited to multi-contributor or compliance-heavy release processes.
Con: meaningful overhead (extra tag, extra pipeline run, manual promotion step) for a single-maintainer project where no second approver exists.
Con: the automatic intermediate/nightly build already provides most of the same dry-run value at lower process cost.

Option 3 — Trigger-tiered pipeline with build-once and debug/release tiers (chosen)¶

Pro: fast feedback on the common path; full validation concentrated at merge to main, before any tag is cut.
Pro: build-once principle, extended across pipeline runs, guarantees the tested and shipped artifacts are identical and makes tagging a low-risk promotion.
Pro: generalizes across C++, Go, and Java without prescribing provider-specific mechanics.
Con: more structure to document and keep the language templates aligned with than a flat pipeline; requires registry promotion tooling (see "Negative Consequences / Risks").

Links¶

specs/guidelines/ci_cd.md — CI/CD guideline (stage order, coverage gate, retention rules) to be updated to match this ADR
specs/guidelines/tool_chain.md — toolchain/image pinning rules referenced by the Build stage
specs/guidelines/containerization.md — distroless/multi-stage image rules referenced by the Package & Publish/Deploy stage
specs/guidelines/monitoring.md — observability guidance referenced by the production debugging escalation path
specs/guidelines/git.md — branch protection, squash-merge/rebase, and the git-cliff CHANGELOG.md rule that the Merge-to-main precondition and "Generate Changelog" job rely on
specs/guidelines/security.md — least-privilege service credential requirement for the CHANGELOG.md commit-back and for registry re-tagging
specs/architecture/ADRs/ADR-001-cpp_build_directory_layout.md — Debug/Release build directory layout for C++
specs/architecture/ADRs/ADR-003-gitlab_pages_release_docs.md — Publish Docs job trigger (release tags only)
specs/architecture/ADRs/ADR-004-distroless_service_runtime_images.md — release vs. debug-capable runtime image variants
internal/templating/cpp/files/Makefile.d/build_cpp.tmpl, internal/templating/cpp/files/Makefile.d/build_cpp_library.tmpl — existing Debug/Release make targets this ADR generalizes across languages
internal/templating/cpp/files/service/image/Dockerfile.tmpl — existing release/debug Docker stage split for C++ services