September 8, 2006
Tips on Using Code Coverage in Visual Studio 2005
One of the things that really helps to leverage unit testing is the idea of "code coverage". That is what areas of my code are not covered by one or more unit tests. Well, also integrated with the unit testing framework that ships with some editions of Visual Studio 2005 is Code Coverage. Furthermore, it's analysis is detailed down to the sub-method level, indicating which portions of methods, not just the methods themselves, are not being tested.
Depending on your project setup, however, you can run into some fairly obscure problems getting it up an running. I learned about this the hard way and so I thought I'd take you through what I did to get it running just in case you ever bump into the same thing I have. This post should serve as a good overview/guide to Code Coverage, as well.
First, to setup the execution of your unit tests to run the Code Coverage analysis, you need to modify the .testrunconfig file that is in your solution. This file is added to your solution upon creating and adding your first unit test and unit test project. When you double click on your .testrunconfig file within Visual Studio, you'll get a dialog box that looks similar to this:
Select the Code Coverage item from the ListBox on the left to see this:
Make sure and select the assembly that you want to profile for test coverage. If the assembly is strongly named, then provide the path to the strong name key file to resign the assembly.
Here is where I ran into problems. After you close this dialog you should be ready to run your unit tests doing two things, 1) testing your code, and 2) profiling the assembly against which you are testing to see how complete you test coverage is. However, when I did this I got this result in the Code Coverage results pane:
How could this be? I set breakpoints and could tell that my code was getting exercised. I thought maybe I should be pointing to the local test results folder that is created for each instance of the test run since I saw that it copied the binaries over there. So I set a breakpoint in my code again and this time opened the Modules window under Debug / Windows, or you could just use the hotkey of Ctrl+Alt+U.
This window allows you to inspect the path of all the resources currently loaded by the debug process:
Eureka! I found the problem. My assembly, AminoSoftware.Ebcdic.dll, that I am trying to profile is located in the GAC. This reminded me that another project that references this assembly, a custom SQL 2005 SSIS component, requires that all referenced assemblies be located in the GAC. So to meet this requirement there had been a post build step in the class library project to uninstall the previous build from the GAC and install the freshly built one.
So, to work around this, I simply wrote a BAT file to uninstall it from the GAC prior to running Code Analysis back in the *.testrunconfig setup dialog:
Now, when I execute the tests I get some pretty sweet Code Coverage metrics:
If this wasn't interesting enough, you can double-click on these individual methods to get highlighting of which blocks within the method are not being exercised by the unit tests:
Those two lines, 70 and 75, are not being tested for. In other words, I have a unit test that tests the Equals method but only testing the Value property. In my tests I am not passing null for the parameter nor am I passing a different type, so those blocks never are tested for accuracy.
Now this is a simple example, but hopefully, the simplicity allows you to see the power of these tools.