Skip to content

Capability 012: Go Builds and Coverage

Status: Accepted

Context

The test runner instruments only the package whose tests run, unless it is told which packages to count. A scenario in test/bdd/ that calls a use case would otherwise count for test/bdd/ alone, and the use case would appear uncovered. A scenario of a Go tool starts the executable as a separate process, which counts only if it is built with instrumentation and writes its counts where the test runner writes them.

The toolchain keeps the compiled packages in its build cache, keyed by the build flags. Builds with different flags therefore never share compiled code, and no build needs a directory of its own to stay separate. The debug and the release executable differ by name only, so neither replaces the other. This capability defines the builds and how the Go profiles measure the figure of capability 011.

References

Details

Builds:

Build Output Configuration Used by
Debug build/<artifact>-debug, none for a library Debug symbols, no optimization, no instrumentation build, run
Release build/<artifact>, none for a library Stripped, no instrumentation release-build
Scenario executable of a tool A temporary directory of the scenario run As Debug, instrumented when coverage runs bdd-test, coverage

The container images build their release executable on their own and are not affected.

Coverage:

Aspect Value
Metric Statements
Counted The tests and the scenarios, each for every package of the module
Excluded cmd/
Scenario executable Writes its counts into the coverage directory of the same coverage run

Functional Requirements

  1. coverage counts every test and scenario for every package of the module, except the excluded ones (see Details).
  2. With coverage, the scenarios of a Go tool start an instrumented executable that adds its counts to the same figure. With bdd-test, they start an executable without instrumentation.
  3. build and release-build produce executables without coverage instrumentation, under the names of Details.

Non-Functional Requirements

None.

Technical Requirements

  1. The Go toolchain builds, go test runs the programmed tests and the scenarios, Godog binds the scenarios, and go tool covdata merges the counts of the scenario executable. The testing guideline prescribes Godog. The rest is the toolchain of the language.

Error Handling

None.

Out of Scope

  • What capability 011 defines for every language
  • The race detector, which test and coverage already use

Acceptance Criteria

  1. Every Go profile: coverage names the packages of the module except cmd/ as the packages to count.
  2. Go tool: the scenario harness builds an instrumented executable when coverage runs, and one without instrumentation otherwise.
  3. Go service and Go tool: build writes build/<artifact>-debug, and release-build writes build/<artifact>. Every Go profile: neither passes a coverage flag.
  4. Generated Go library, Go service, and Go tool, a function reached only by a scenario: make coverage reports it as covered (manual test test/manual/generated_coverage.md).

Decisions

  1. Count every package, not the package of the test. A scenario reaches several packages by design. Counting it only for test/bdd/ would push the developer to write the same test twice. Consequence: a package without tests of its own can reach its figure through scenarios alone.
  2. The harness of a tool builds its own executable. It already does so for bdd-test, so coverage only changes how it builds. An executable from build would have to be built twice, once with instrumentation. Consequence: the instrumented executable lives only as long as the scenario run.
  3. The debug executable carries the suffix -debug. One directory build/ suffices when the names differ, and Go has no convention for build directories per configuration. Consequence: the release executable keeps the plain name that the image and the documentation use.
  4. Statements, not lines. Go's tools count statements. A conversion to lines would need a further tool and change nothing in substance. Consequence: the figure of a Go artifact is not exactly comparable with a line figure.