# Google Test/Mock, Benchmark, Coverage

Nivå: intermediate | Målgrupp: C++ programmerare | Förkunskaper: Stor vana att programmera i C++
https://www.ribomation.se/programmerings-kurser/gtest-gmock-gbench/

Skriv inte en rad produktionskod om du inte skriver testkod samtidigt. Denna kurs
lär dig allt du behöver veta om att skriva enhetstester med Google Test/Mock.
Plus mäta tidsåtgång med Google Benchmark, samt kontrollera hur mycket kodtäckning
du har med gcoc/lcov/genhtml.

I det första avsnittet behandlar vi Google Test fullständigt, inklusive hur man installerar
GTest & GMock (del av samma repo) och Hamcrest matchers (del av GMock).

I det andra avsnittet behandlar vi mock objects delen av GTest/GMock-repo.
Du får lära dig att skapa och konfigurera mock objects, så att ett dylikt kan
matcha förväntade anrop och förväntade argument.

I det tredje avsnittet behandlar vi Google Benchmark, som är relaterat med GTest
men utgör ett helt eget ramverk. Med GBench kan du skriva mikro-benchmarks att 
upprätthålla och mäta prestanda för tidskritiska funktioner i ditt system eller 
för att jämföra olika implementeringar av samma funktionalitet.

I det fjärde avslutande avsnittet tar vi upp ämnet testtäckning (test coverage), 
som involverar instrumentering av test-programmet under kompileringen och kör 
olika aggregeringsverktyg för att producera en täckningsrapport, antingen textuell
med `gcov` eller som en HTML-rapport med `lcov/genhtml`.

- Writing test functions with _Google Test_
- Understand how to use different forms of checks, based on target data types
- Writing user-defined predicate tests
- Know have to use Hamcrest matchers
- Using matchers for text comparison
- Using matchers for STL containers
- Know how to write failure tests for various forms of program crashed
- Writing tests with associated external test data
- Designing value-parameterized tests
- Creating and configure mock objects with _Google Mock_
- Writing micro-benchmarks with _Google Benchmark_
- Know how create data sets fo benchmarks
- Understand how to configure GCC for test coverage measurement
- Using `gcov` to generate text reports
- Using `lcov` and `genhtml` to generate multi-page HTML coverage reports 

#### What is a Unit Test?
- Why do we write bugs?
- It's not always about bugs
- Time to market
- Cost of fixing bugs
- Eric Gamma & Kent Beck
- What is a unit test?
- Other forms of tests
- Structure of a unit test
- Check one aspect at a time
- The acronym F.I.R.S.T.
- Brief about test-driven development (TDD)

### Google Test
In this first section we cover Google Test completely, including how to install
 GTest & GMock (part of same repo) and matchers (part of GMock).

#### Installation of GTest & GMock
- Overview of the package
- GitHub repo
- Steps and alternatives to install it
- (a) Download and build it yourself
- GTest directories
- How to compile a test executable
- Global installation
- Simple CMake file to locate GTest
- (b) Let CMake download and build it for you
- Using FetchContent
- A very simple unit test
- Test runner driver
- User-defined test driver
- _Exercise_: Use CMake to download GTest and write a simple test

#### Test Functions
- GTest is based on CPP macros
- The `TEST(group, name)` macro
- Generated function, as shown by `nm`
- How to inspect generated sources
- Failing a check, how it looks like
- Expectations vs. Assertions
- Failing a set of expections
- Failing a set of assertions
- Boolean checks
- Relational checks
- _Exercise_: Write tests for a simple Person class

#### More about Checks
- Checks fo native strings (C string)
- Comparing C++ strings vs. C strings
- Floating-point checks
- Floating-point operations - problems to be aware of
- Floating-point checks
- Custom error messages
- Predicate checks
- Sample predicate checks
- Explicit failure or success
- Usage of `ADD_FAILURE()`
- Printing values
- _Exercise_: C-style upper-case function
- _Exercise_: Approximate sin(x)
- _Exercise_: Character-unit palindrome

#### Matchers
- What is a matcher?
- How to use matchers
- Build instructions
- Basic usage of `EXPECT_THAT()`
- Relational matchers
- Matchers for primitive expressions
- Text matchers
- Regex matchers
- Floating-point matchers
- Composition matchers
- Container matchers
- Class/struct member matchers
- User-defined matchers
- _Exercise_: Rewrite your previous test, now using matchers instead

#### Failure Tests
- What is a failure test?
- Exception tests
- Checking for a thrown exception
- Checking for the wrong exception
- How to check for abnormal program termination
- _Death test_ checks
- _Death test_ predicates
- Checking for exit code
- Checking for internal signals
- Checking for null-pointer termination (`SEGV`)
- Checking for division by zero termination (`FPE`)
- Checking for invalid call to delete termination (`ABRT`)
- Failed death test
- _Exercise_: Implement and test factorial (n!), that throws fdor negative arguments
- _Exercise_: Implement and test factorial (n!), that terminates the program

#### Test Data & Life-Cycle Functions
- What is test-data?
- Using a fixture class
- Life-cycle methods
- Before/after every test-case
- Before/after a test-group
- Before/after a test-suite
- (a) Implement `testing::Environment `using a global variable 
- (b) Implement `testing::Environment` using your own `main()`
- Showing all life-cycle methods in one program
- _Exercise_: Implement an write tests for a word count program that read text files

#### Value-Parameterized Tests
- What is a Value-Parameterized Test?
- How to create a value-parameterized test
- Value generators
- Value from a container
- Values from a numeric range
- Boolean values
- How to combine parameters with expected result
- Advanced value generators
- Cartesian product tuple values
- User-defined value generators
- _Exercise_: Implement and write tests for a left padding function

#### Type-Parameterized Tests
- What is a Type-Parameterized Test?
- How to create a type-parameterized test
- Target container with regular test
- Type fixture and type parameters
- Test macro functions
- _Exercise_: Complete the provided test suite

#### Command-Line Usage
- Running GTest on the command-line
- Selecting tests
- Sample filters
- Repeated tests
- Disabled tests
- Skipping elapsed time
- Skipping coloured lines
- Generate XML reports for post-processing
- _Exercise_: Investigate various CLI options, using your own tests

### Mock Objects
In this section we cover the mock objects part of the GTest/GMock repo.

#### Test Doubles Terminology
- What is and why do we need test doubles?
- Test double types
- Stub
- Spy
- Fake
- Mock

#### Using GMock
- Brief about installation
- Prerequisites on the target system
- Sample C++ interface
- Test-target system
- Test-target method
- How to generate a mock class
- Sample mock class
- A minimal unit test
- Default behaviour of a mock object
- How to hide "Uninteresting" warning printouts, using `NiceMock<>`
- How to do the opposite with `StrictMock<>`
- Be aware of limitations of the pre-processor (CPP)
- How to compensate for the CPP parsing limitation
- How to specify expected behaviour of mocked methods
- Simple configuration
- Argument matchers
- Return values
- Actions
- Invocation cardinality
- Cardinality failures
- _Exercise_: Write mock tests for the provided ATM system

#### Other Testing Frameworks
- Catch2, version 3
- Doctest
- CppUnit
- List of C/C++ testing frameworks

### Benchmarking
In this section we cover Google Benchmark, which related by still a separate framework from GTest/GMock.
With GBench you can write micro-benchmarks to keep track of performance for time-critical functions
in your system or to compare different implementations of the same functionality.

#### What is Benchmarking?
- Benchmarking - a definition
- Benchmarking - a motivation
- Types of benchmarks
- Benchmarking metrics
- Benchmarking tactics
- Pitfalls to be aware of
- A very simple benchmark
- Shortcomings of this implementation
- A slightly less simple benchmark
- The `benchmark()` function
- The `warmup()` function
- The `measure()` function
- Shortcomings of the second implementation
- _Exercise_: Implement the second version and populate a `std::vector<unsigned long>`

#### Installation of Google Benchmark
- The GitHub repo of GBench
- Ways of installing GBench
- (a) Using FetchContent in CMake
- Sample JetBrains CLion setup
- (b) Download and build it yourself
- (1-2) Download & unpack
- (3-5) Build & install
- (6) Usage of `find_package()` in CMake
- (7) How to compile on the command-line
- Sample simple benchmark
- _Exercise_: Reimplement your previous benchmark, using GBench

#### Basic Usage of GBench
- Benchmark function(s)
- Registration of a measurement function
- How to provide the `main()` function
- What happens if you don't prevent compiler optimizations?
- Marking the variable as `volatile`
- Marking the variable with `DoNotOptimize()`
- What does `DoNotOptimize()` do?
- What is clobbering the memory?
- Using `ClobberMemory()`
- Excluding timings for a code snippet
- Comparative benchmarking
- Sample comparative benchmark
- _Exercise_: Benchmark `std::sort()`

#### Command-Line Usage of GBench
- Getting help info
- CLI <--> ENV
- Double-check the documentation, because not all is up-to-date
- Setting verbosity level
- Setting the time unit
- Available output formats
- The _console_ format
- Data --> `stdout` and context --> `stderr`
- The _json_ format
- The _csv_ format
- Adding extra context as _key=value_ pairs
- Sending data to an output file
- Re-run a subset of a benchmark suite
- Repeatedly re-run a benchmark
- Mixed output of results and stats
- How to suppress the individual result rows
- Aggregates to `stdout`, but full report to file
- _Exercise_: Investigating the command-line options

#### Benchmark Arguments
- Change of time unit
- Benchmark arguments forms
- Explicit arguments with `Arg()`
- Sparse range arguments with `Range()`
- Change of multiplicator with `RangeMultiplicator()` 
- Dense range arguments with `DenseRange()`
- Composite arguments
- Composite arguments range
- Cartesian product of arguments
- Cartesian product of arguments, with generators
- Arbitrary arguments
- Sample usage of a parameter pack function
- Lice-cycle functions
- Setup & teardown hooks
- Fixtures
- User-defined counters
- _Exercise_: Investigate `std::sort()` with various collection sizes
- _Exercise_: Write a benchmark the reads a file using the provided WordIterator and finds the shortest and longest words

### Coverage
In this section we cover the topic of test coverage, which involves instrumenting the test executable
in compile-time and running various aggregating tools to produce a coverage report.

#### What is Test Coverage?
- What do we mean with coverage?
- What is its purpose?
- Types of test coverage
- Function coverage
- Line coverage
- Branch coverage
- Path coverage
- What are the benefits of high coverage percentage?
- Setting realistic coverage goals
- Common misconceptions of test coverage

#### Using GCOV
- Capturing tools for GCC
- How to compile with instrumentation
- What does option `--coverage` do?
- Files generated during compilation
- File generated - an overview
- Simple single-file project - source code
- Simple single-file project - commands
- Simple single-file project - output
- Coverage of project with several files
- Simple multi-file project - class Person
- Simple multi-file project - person generator
- Simple multi-file project - person app
- Simple multi-file project - build.sh
- Simple multi-file project - running the build
- Simple multi-file project - sample output `*.gcov` files
- Setting coloured lines
- _Exercise_: Generate coverage for the person app

#### Usijng LCOV
- What is LCOV?
- Installation of LCOV
- How to download and build LCOV
- Basic usage of this tool
- Simple multi-file and multi-folder project
- File structure
- Application build
- Application run
- Basic usage of program `lcov`
- Contens of `*.info` files
- Sources processed by `lcov`
- How to include just your project files - i.e., exclude stdlib++
- Basic invocation of `lcov`
- Reduce the log output
- Basic usage of program `genhtml`
- How to open an HTML file from the command-line or a shell script
- Using `wslview`
- Sumamry of running `lcov` and `genhtml`
- Remarks about the report HTML & CSS
- _Exercise_: Perform coverage of the project _Just RESTing_