Measure Coverage

Howto BullsEye - Quick...

Before we start, understand a few facts


  • BullsEye functions by inserting it's own compilers and linkers instead of those choosen by your environment. That way they can flag the code, and count if the unittest reaches the flag.
  • BullsEye defaults to use the global Coverage file selected by the Environment variable COVFILE. Most of the utilities documents support for selecting at specific covfile - that doesn't work too convincing in the version we are using.
  • If nothing is selected - meaning deliberately added to the COVFILE - BullsEye will add any file that the build or unittest is accessing. Which in turn will produce uninteresting coverage percentage numbers.
  • There will be no coverage measurement unless the built unittest has been executed, which means that the unittest should be freshly built, and as the unittest exercises a module, that module should also be built with coverage.
  • BullsEye seems to keep a number of results in memory, turning it off seems to make it flush to %COVFILE%

As per these facts we need a procedure that does the following.

Select coverage file

Change the data of Environment value COVFILE, i.e.

set COVFILE=%SYSTEMDRIVE%\mycov.cov

It is advised not to change the value again, let it be the one you use untill the session finishes, but the file you select should be for this session only.

The steps


If you are measuring several modules, or have multiple unittests, your code can loop from this point.
****************** Loop start
First parse the project file, so we get all the relevant files that are included in the project / module that we are going to unittest and get coverage for.
Save result to a temporary file which we will use later, in this example we call that one %tempfile%.

BullsEye Coverage file has it's own internal format, so lets create one now (this is a new  file, right...)

covselect --create --deleteAll --no-banner --quiet

Turn on Coverage Building (so BullsEye can injects it's compilers and linkers)

cov01 --on --quiet --no-banner

Add the files you parsed earlier and saved to a temp file (files can be added one by one, by that is slower)

covselect --import %tempfile% --create --no-banner

Now build the project, the unittest and run the unittest....

When done results are in the file selected by %COVFILE%

Turn off Coverage Building

cov01 --off --quiet --no-banner

(This saves whatever findings bullseye still has in memory....)

copy the covfile to another location

##optional
Before you do another measurement, you can export the results to your likings.
Result export utilies are:

covclass
covbr

feel free to read doc on the utils yourself - they do tell about themselves if you just call them in the command prompt


The delete %COVFILE%, a new will be initialized in beginning of loop

Loop - get next project to parse.

******************

When loop is done you can merge all the covfiles you created into one big with the covmerge utility

covmerge





Comments