Skip to content

Testing Guidelines

Principles

  • Test cases are independent — no shared mutable state between tests.
  • Test public interfaces, not internal implementation.
  • Unit-test the happy path and all boundary cases.
  • Tests are deterministic — no sleep, no wall-clock time, no unfixed random seeds.
  • Inject clocks, UUID generators etc. that are controllable by the test driver
  • No network or file-system access in unit tests.

Test Integrity

  • A failing test signals a defect in the code under test, not in the test — fix the code, not the test.
  • Only change a test when the test itself is proven wrong (e.g. it encodes an outdated requirement or asserts an incorrect expected value); state that reasoning explicitly when you do.
  • Never weaken an assertion, delete a test, or add a skip/ignore annotation solely to turn a red run green.

Test Kinds

A developer writes three kinds of automated tests. All of them run from the IDE and in the pipeline. Fewer tests of each kind as the list goes on, as the test pyramid suggests.

  • Unit test — checks the smallest unit, such as one function, often without a mock.
  • Component test — checks units working together, such as a use case, with mocks at the layer boundaries.
  • Integrative test — a component test with one real dependency, such as a database in a container. It is the narrow integration test of Martin Fowler.

The integration test of a deployed image in a real environment is not a developer test. End-to-end tests cover critical user journeys only.

A test is programmed (JUnit, Catch2, Testify, Vitest) or a scenario in a feature file. Both forms suit every kind. Scenarios suit the more complex component and integrative tests, whose setup and flow they make readable.

Language Unit test Component test Integrative test
C++ *_test.cpp *_ct_test.cpp, Catch2 tag [ct] *_it_test.cpp, Catch2 tag [it]
Go *_test.go *_ct_test.go *_it_test.go, build constraint integrative
Java *Test.java *CT.java, @Tag("CT") *IT.java, @Tag("IT")
TypeScript *.test.ts(x) *.ct.test.ts(x) *.it.test.ts(x)

A suffix works in every framework and shows the kind in the file list. Add the tag where the framework has one. A scenario lives in test/bdd/features/, so its directory marks it. A scenario that is an integrative test carries the Gherkin tag @integrative.

BDD

  • Acceptance criteria are covered by BDD scenarios.
  • Structure: Given / When / Then.
  • Preferred frameworks: Godog (Go), Cucumber + JUnit 5 (Java), pytest-bdd (C++ acceptance tier).
  • Every language reads its Gherkin scenarios directly from test/bdd/features/ — the single source of truth, doubling as living documentation. No language copies or symlinks feature files into its own test tree.
  • C++: a scenario (or step) with no automated pytest-bdd binding by design — structurally out of reach for this test tier, not merely unimplemented — is tagged @no_step_definition and excluded via the generated pytest.ini; document the specific reason next to the scenario or in test/bdd/README.md. It does not mean "untested": verification may still happen at another tier.
  • The scenarios serve the developers. They are component tests unless tagged @integrative, and use mocks instead of real infrastructure to stay fast and focused on the use cases.

Coverage

make coverage reports one figure for the unit and component tests, programmed ones and scenarios alike. Integrative tests do not count. Unit and component tests aim at particular code, while an integrative test checks a feature and passes through much code without checking it.

Integrative Tests

Some correctness (ORM/schema mapping, migration behavior) can only be verified against a real dependency — mocking it would defeat the point. Keep such tests out of the default targets, so the fast unit/component loop never needs Docker or any other external service:

  • Run them in a target of their own, make integrative-test. Never let them run as part of make test, make bdd-test, or make coverage.
  • Java: pom.xml names the includes of Surefire (*Test.java, *CT.java) and Failsafe (the Cucumber suite) explicitly, so neither picks up *IT.java by default. spring-boot-starter-parent binds Failsafe for every child POM, which would otherwise run *IT.java during mvn verify. The integrative target selects the tests by @Tag("IT").
  • Not every JUnit 5 extension honors @Tag: some libraries (e.g. ArchUnit's archunit-junit5) register their own JUnit Platform TestEngine that does not expose @Tag to Surefire's groups/excludedGroups filter (confirmed empirically — -Dgroups=<tag> produced zero executions of an ArchUnit test tagged that way). For a test whose engine doesn't honor tags, fall back to filename-based Surefire <excludes> for the default run plus an explicit -Dtest=<ClassName> for its dedicated target, instead of assuming tags work universally.
  • Go: the build constraint integrative keeps go test ./... from compiling the tests; the target passes -tags integrative. Godog selects @integrative scenarios with its tag filter.
  • C++: Catch2 selects [it], pytest selects the integrative marker derived from the Gherkin tag.
  • TypeScript: Vitest and Playwright select the tests by the .it.test. suffix and the @integrative tag.
  • Exclude the code only integrative tests exercise (e.g. JPA entities, repository adapters) from the coverage gate, so the threshold (see ci_cd.md) stays reachable without Docker.

Mocking

  • Mock only at layer boundaries (e.g., infra interfaces defined in usecase).
  • Never mock domain or use-case logic — test it directly.

Test Data

  • Use builders or factories; avoid large inline literals.
  • Never depend on test execution order.
  • Every value passed to a Gherkin step must be asserted by a Then or exercise a business rule. If a domain field only exists to satisfy this pattern with no real behavior behind it, that's a YAGNI signal — remove the field instead of keeping it around with placeholder test data.

Directives

  • A failing test means the code is wrong, not the test. Change a test only when it is proven wrong, and say so; never weaken, delete, or skip a test to turn a run green
  • Test public interfaces, not the implementation; tests are independent and deterministic
  • No sleep, wall-clock time, or unfixed random seeds; inject clocks and ID generators
  • No network or file-system access in unit tests
  • Mock only at architectural layer boundaries; never mock domain logic
  • Every automatically verifiable acceptance criterion has a BDD scenario: Godog for Go, Cucumber with JUnit for Java, pytest-bdd for C++; Catch2 covers C++ unit tests
  • Name tests by kind: no suffix for unit tests, CT for component tests, IT for integrative tests; add the framework's tag where it has one, and @integrative to an integrative scenario
  • BDD scenarios serve developers, not end-to-end testing; they are component tests unless tagged @integrative and may use mocks instead of real infrastructure
  • make coverage counts unit and component tests, programmed and scenarios; never integrative tests
  • End-to-end tests cover critical user journeys only
  • Build test data with builders or factories; never depend on test order
  • Every Gherkin step parameter is asserted by a Then step or exercises a rule; remove a field that exists only to fill a step
  • Integrative tests, such as with Testcontainers, run in make integrative-test, never in make test, make bdd-test, or make coverage; exclude the code only they cover from the coverage gate