Skip to content

Capability 008: C++ Builds and Coverage

Status: Accepted

Context

A C++ artifact is built locally for three purposes:

  • a debug build for the IDE and the debugger, which may also run on a target to investigate a problem,
  • a coverage-instrumented build for the automated tests, and
  • an optimized release build. Each has its own compiler flags.

The build system keeps the flags of a build directory, so two purposes sharing one directory leak flags into each other: the debug executable writes coverage data wherever it runs, and stale coverage data from an earlier run breaks the tests after a code change.

The compiler instruments the coverage build. Every process of that build adds its counts to the coverage data next to its object files. A scenario that starts the executable of the coverage build therefore counts like a programmed test. This capability defines the builds and how the C++ profiles measure the figure of capability 011.

References

Details

Builds:

Build Directory Configuration Used by
Debug build/debug, build/Debug for a library Debug symbols, no optimization, no instrumentation build, test, run, lint
Coverage build/coverage As Debug, plus coverage instrumentation coverage-build, bdd-test, coverage
Release build/release, build/Release for a library Optimized, no debug symbols, no instrumentation release-build, release-test

The coverage build uses the dependencies of the debug build. The container images build their release executable on their own and are not affected.

Coverage:

Aspect Value
Metric Lines
Counted The programmed tests and the scenarios, both run against the coverage build
Excluded src/main.cpp, test/, the build directories, and the dependency cache

Functional Requirements

  1. Every C++ profile builds each build of Details in its own directory, and only the targets listed there use it.
  2. No target adds coverage instrumentation to the debug or the release build.
  3. coverage-build deletes the coverage data of earlier runs, so that bdd-test and coverage count the current run only.
  4. coverage runs the programmed tests and bdd-test against the coverage build and reports their counts together, as Details defines.
  5. No C++ profile has a release-coverage target or pipeline job. release-test runs the tests against the release build.

Non-Functional Requirements

None.

Technical Requirements

  1. CMake with Conan builds, Catch2 runs the programmed tests, pytest-bdd the scenarios, and gcovr reports the coverage. The testing guideline prescribes the test tools, and the C++ CI images provide all of them.

Error Handling

None.

Out of Scope

  • Sanitizers and their build option
  • What capability 011 defines for every language

Acceptance Criteria

  1. Every C++ profile: the Makefile passes coverage flags to the build system only in the recipe of coverage-build, and only for build/coverage.
  2. Every C++ profile: bdd-test and coverage read build/coverage. build, test, and run read the debug directory.
  3. Every C++ profile: the recipe of coverage-build deletes the coverage data (.gcda files) below build/coverage, and coverage searches coverage data in build/coverage only.
  4. Every C++ profile: coverage depends on bdd-test and excludes the paths of Details.
  5. Every C++ profile on both forges: no release-coverage target in the Makefile, no release-coverage job or step in the pipeline.
  6. Generated C++ service, make coverage, a source file changed, make coverage again: both runs pass. Afterwards make build produces an executable without coverage instrumentation (manual test test/manual/generated_coverage.md).
  7. Generated C++ service, a function reached only by a scenario: make coverage reports it as covered (manual test test/manual/generated_coverage.md).

Decisions

  1. Three builds in three directories, not two. Debug and coverage serve different ends: an instrumented executable writes coverage data wherever it runs, which is unwanted where the debug executable investigates a problem. Consequence: a developer who both tests and debugs builds the sources twice.
  2. No coverage of the release build. release-test already shows that the tests pass with optimization. A second coverage figure of optimized code, whose lines the compiler merges and reorders, adds nothing.
  3. The scenarios run the coverage build. Its executable counts wherever it runs, so no step collects counts from the scenarios. Consequence: bdd-test builds the coverage build first, and coverage depends on bdd-test.