3.6 Code Instrumentation and Test Trace Management
Instrumenting C code
The Aristotle Code Instrumenter generates
instrumented C source code for a given source code file.
At present, two versions of the instrumenter are available:
the first, il-st, generates instrumented programs to collect
statement coverage information, the second, il-bt, generates instrumented
programs to collect branch coverage information.
You can invoke the instrumenter versions from the command
line using the following commands:
il-st-2 <source-filename>
il-bt-2 <source-filename>
In this invocation, <source-filename> refers to a C program
that you wish to instrument.
The output will be in a file nameed <source-filename>.int.c in the current
working directory. See the examples below.
The commands take the named source file, and using
analysis information found in the database
create a new version of the source file that, when
executed, reports statements or branches
traversed during that execution.
Before you can run the instrumenters on a source file, you
must run the source file through CFE.
Compiling and Linking instrumented C code
To instrument a program, the instrumenter places calls into the
program that reference routines in Aristotle libraries.
These routines create bit vectors that hold coverage information,
set bits in these vectors when program components are executed,
and write out coverage information when programs terminate.
To compile and link an instrumented file you must
ensure that the compiler can access the Aristotle header files and
Aristotle libraries that are required.
Suppose you wish to compile and link file "foo.int.c", which is an
instrumented version of the source file "foo.c".
Use the following command to compile the file into object
file "foo.inst.o":
gcc -g -I<Aristotle>/headers -c -o foo.int.o foo.int.c
where <Aristotle> names the location of your
Aristotle installation. The GNU gcc compiler is used
for illustration.
You should be able to use cc, also.
To link the object file "foo.int.o" that you just created,
and output an executable file "foo.int.exe", issue one
of the following commands:
gcc -g -o foo.int.exe foo.int.o -L<Aristotle>/lib/ -lIPF_st -lglobalfunc
gcc -g -o foo.int.exe foo.int.o -L<Aristotle>/lib/ -lIPF_bt -lglobalfunc
where again, <Aristotle> names the location of your
Aristotle installation. The only difference between the
two commands is the library specified by the "-l" option.
The first command specifies the "IPF_st" library: use
this when you're dealing with a file instrumented for statement coverage.
The second command specifies the "IPF_bt" library: use
this when you're dealing with a file instrumented for branch coverage.
Executing an instrumented program
After you have compiled and linked your instrumented executable,
execute it just as you would execute a non-instrumented executable.
Given an instrumented executable created by instrumenting
source file "foo.c", when you run that executable,
it writes a file named "foo.c.tr" to the database directory.
Each time you run the instrumented executable, it writes
a new version of this file, overwriting any version previously
present in the database. Thus, to collect a number of
coverage files, after running your executable you must
save the file that was created in the database directory.
One approach is to create a "test coverage database directory"
and move or copy each trace file there.
This will enable you to use the test
history management and viewing tools described below.
Viewing coverage information output by instrumented programs
It should be noted that the term "trace file" is used loosely below to mean
a file containing either branch coverage information or statement coverage
information. They are not actually complete traces of execution.
You can view a single coverage file using Aristotle's
trace viewer, as follows:
tr_printer <trace-file-name> [<source-file>|"SHORT"] [<output-filename>]
In this invocation, <trace-file-name> is the
name of the coverage file you wish to view,
<source-file>, if specified, is the name of the
source file whose coverage information you are viewing,
and <output-filename> is an optional argument
that names the file to which the viewer should
write information.
Unlike other printers, the tr_printer
does not assume the coverage file is
in the database, and does not prepend
the path to the database directory to
the file name that you give; thus,
you must give the full pathname (or relative
pathname) to the file.
The second (optional) argument may be a source
file name, or the string "SHORT".
In the former case, the printer prints
a table of statements or branches, indicating
for each whether the statement or branch
was marked "hit".
To invoke the trace viewer in this fashion,
you must ensure that the ".cf" and ".map"
files that correspond to the
instrumented source file are present in your database.
If you specify "SHORT" (the default
if you omit this parameter entirely)
the printer simply types a list of
object ids for objects that were
hit.
If you omit the output filename,
output is directed to the screen.
There are ".tr" file handler routines
available, if you wish to access coverage
files from a program. See Section 2.1
of this manual for details.
Building test histories
A single coverage file shows you, for one test,
which objects (statements or branches) in a program
were traversed by that test.
More useful, typically, is information that
lists, for each test in a test suite,
the objects traversed by that test.
We call this a test history,
and provide utilities for creating
and viewing test history information.
You create a test history for a collection
of coverage files. To create the history you must
first create a set of coverage files, as described above, by saving them to some
location so they will not be overwritten by each test execution. Do this by
running your tests on your instrumented
executable, and placing the coverage files in
some directory -- we'll refer to this
directory as your "test coverage database directory".
The names you give to your coverage files are
important: read on to see why.
Run the test history builder by issuing the command:
th_builder <test_dir> <test_history_file> <test_prog>
where <test_dir> is your test coverage database
directory, <test_history_file> names the test
history file you wish to create, and <test_prog>
names the C source file that you instrumented.
The th_builder locates every ".tr" file
in <test_dir> and creates a test history
file that lists, for each test,
the objects (statements or branches)
that were traversed by that test.
To locate coverage files, the th_builder issues
the UNIX command "ls -1 *.tr", piping its
output through the command "sort -n", in <test_dir>,
and constructs a history that accounts for every
file thus listed, in the order in which it is
listed. The th_builder assigns integer identifiers
to each coverage file thus listed, beginning with 0.
If you wish your tests to be assigned
integer identifiers that you can easily
map to their associated coverage information, you
should name your coverage files "0.tr, 1.tr,...,n.tr".
(Thus, you should consider assigning integer
identifiers 0,1,...,n to your tests).
Viewing test histories
To view a test history, issue the command:
th_printer <test_history_file> <prog_name> ["COUNT"|"LIST"] [<output_file>]
where <test_history_file> names your
test history file, <prog_name> names
the source code file that you instrumented,
the third (optional) parameter contains
the string "COUNT" or the string "LIST",
and the fourth (optional) parameter names
the output destination for the printer.
Unlike other printers, the th_printer
does not assume the test history file is
in the database, and does not prepend
the path to the database directory to
the coverage file name that you give; thus,
you must give the full pathname (or relative
pathname) to the history file.
The third (optional) argument
determines the format of the data
output by the viewer.
A "COUNT" lists objects, the
number of tests that traverse them,
and the percentage of total tests
that this number represents.
A "LIST" lists objects,
and the identifiers (these being
the identifiers assigned to coverage
files by the test history builder,
by the procedure detailed above)
of tests that traversed them.
The default value of this parameter,
if you omit it, is "LIST".
If you omit the output filename,
output is directed to the screen.
There are test history file handler routines
available, if you wish to access test history file
information from a program. See
Section 2.1
of this manual for details.
Building and Viewing Edge Test Histories
Aristotle can build an edge test history from an
existing branch test history file if control flow information
is available in the database.
To build an edge test history, use the command:
et_builder <program_name> <input_history> <output_history>
where <input_history> is a previously built branch test history
file. If no output_history file is specified, the output will be displayed
on the terminal. The required control flow information must already exist
in the Aristotle database, or the edge test history will fail.
The edge test history file can be viewed using the th_printer program,
just like for a branch test history file.
