Code Coverage for WebAssembly

This is a general overview of the WasmCov WebAssembly coverage generation method used by Wasmcov. In 2023, generating WASM coverage reports using LLVM-COV wasn’t possible for projects using the WebAssembly VM. This document aims to guide projects using the WebAssembly VM in implementing code coverage functionality.

Step 1: Add Coverage Instrumentation To Your WasmCov Binary

Rust

Minicov provides an easy way to add LLVM instrumentation coverage to Rust projects. This will allow you to get the coverage information.

Set the RUSTFLAGS to:

RUSTFLAGS="-Cinstrument-coverage -Zno-profiler-runtime --emit=llvm-ir"

And then compile your project in debug/develop mode (or as release with LTO disabled).

Note: minicov doesn’t mention --emit=llvm-ir, but it’s needed later.

Step 2: Parsing Raw Coverage Data

Executing your code will result in one or more .profraw files. Merge these into a single .profdata file using LLVM tooling:

llvm-profdata merge -sparse *.profraw -o coverage.profdata

Note: Your llvm-profdata install must match your Rust version (rustc --version --verbose). If they differ, install the correct LLVM version and adjust the command accordingly (e.g. llvm-profdata-17).

Step 3: Generate An Object File

Coverage generation doesn’t work by default because a .wasm binary doesn’t have a __llvm_covmap section, required by llvm-cov to generate the report.

$ llvm-cov-17 show --instr-profile=coverage.profdata our_binary.wasm
error: Failed to load coverage: 'our_binary.wasm': No coverage data found

Rust

To fix this, the IR file our_binary.ll generated by the --emit=llvm-ir flag and located in target/wasm32-unknown-unknown/debug/deps is used.

clang-17 our_binary.ll -Wno-override-module -c

Step 4: Stub WASM Specific Instructions

The above approach won’t work if the project is using instructions only available in WebAssembly, like memory.size or memory.grow.

Fortunately, instructions in our_binary.ll are not needed to generate a coverage report, so you can just remove all of them using a regex.

perl -i -p0e 's/(^define[^\n]*\n).*?^}\s*$/$1start:\n  unreachable\n}\n/gms' our_binary.ll

Step 5: Generate The Coverage Report

The report can now be generated using the object file for function mapping instead of the wasm compile.

llvm-cov-17 show --instr-profile=coverage.profdata our_binary.o --format=html -output-dir=coverage/

While this method utilizes some workarounds, it’s functional. A more streamlined approach would involve integrating WASM file support into llvm-cov. However, this would be more challenging and time-consuming. Thus, this solution is recommended for now.