TestDriven.NET by Jamie Cansdale : NCoverExplorer

TestDriven.Net 2.20: Improved NCover Integration

Jamie Cansdale — Tue, 21 Apr 2009 15:01:54 GMT

In the latest release of TestDriven.Net, you’ll find much improved integration with all versions of NCover:

Support for 64-bit Windows

‘Test With > Coverage’ now works on 64-bit versions of Windows. In previous versions this could be made to work by compiling your test project for x86. The new version will automatically execute your tests in a 32-bit process if a 64-bit version of NCover can’t be found. (There is similar 64-bit Windows support for dotTrace and Team Coverage)

No more ‘Reload coverage file?’ dialog

NCoverExplorer 2.1 & 3.0 will no longer prompt to reload the coverage file. The coverage report will automatically refresh when new data is available. If you’re using NCover 3.0.18 or higher, you won’t be prompted to save a ‘New Project’ when you close NCoverExplorer.

Automatic assembly filtering

If you’re using NCover 3.0, only assemblies that have a corresponding PDB file will appear in the ‘Explorer’ window. I’m hoping this is a good alternative to explicitly naming assemblies that shouldn’t appear in the coverage support.

Support for Typemock Isolator

You can now execute ‘Test With > Coverage’ with NCover 3.0 and Typemock Isolator installed. (Previous versions would fail with the error: ‘Couldn't find TypeMock profiler name for NCover 3.0’.)

Recommended Versions of NCover & Typemock

If you have a commercial license for NCover, make sure you’re using NCover 3.0.18 or later. If you’re using Typemock Isolator, I recommend you use at least version 5.3.0.

If you’d like more timely updates on new releases, you can find me on twitter here.

NCoverExplorer... v1.4.0

Jamie Cansdale — Wed, 10 Oct 2007 18:37:54 GMT

Syndicated from Kiwidude's Geek Spot

I've had this "quietly available" on the download page for a few weeks to gain feedback on any issues and with no known issues outstanding it seems appropriate to more publicly announce it.

A big change in NCoverExplorer 1.4.0 is that it is no longer published under the GPL license. I made this change for a number of reasons; amongst which was the concern that a number of companies have of deploying GPL based tools into their development process. This roadblock has been removed and should no longer prevent your companies from using this toolset.

My original choice of GPL was based on using the ICSharpCode 1.x texteditor control. I have now dropped that component in favour of using the superior (but commercial) Actipro SyntaxEditor control instead. The improvements are not only in stability and features but also visual as you can see below in the code highlighting screenshot. A splitter window is available as also shown.

So why should you upgrade to NCoverExplorer 1.4?

This release is less about adding new features (although there are some to be found) and more about stabilisation for the future. The core model code has been rewritten and better tested. There have been countless code coverage corner cases (say that five times in a row!) arise that should be better catered for. Better performance for .NET 2.0 users, lower resource usage, better error handling and some minor gui quirks addressed.

Some of the "internal" changes have spilled over externally. For instance the configuration file format used by NCoverExplorer.Console with /c has changed to support new features, although I have retained legacy compatibility for those who choose not to upgrade their build scripts immediately.

What about the new features? Well, the ones I did squeeze in were...

Support for the //pm (profiling a process by name) argument in NCover 1.5.8;
Copy command in the source code text area via right-click;
Print preview command in the File menu;
/fc argument for NCoverExplorer.Console to fail a build if the combined total coverage of all assemblies does not meet the minimum specified. This was the original behaviour of the /f command before it was changed to fail if any individual assembly does not meet the minimum;
Coverage exclusions now support the '?' wildcard and multiple '*' wildcards within the expression.
Report xml contains a "totalSeqPoints" attribute as requested for developers writing alternate xsl reports that need to calculate the excluded code at each level;
Update the NAnt/MSBuild task dlls to support the latest changes;
Redesigned the options dialog as shown below;

NCoverExplorer... v1.3.6

Jamie Cansdale — Fri, 06 Apr 2007 11:50:21 GMT

Syndicated from Kiwidude's Geek Spot

Some of you may already have noticed the long awaited release of NCover 1.5.7 yesterday - hoorah! That now being out of the way and in conjunction with some easing of my day job pressure, has made it a good time for me to finally rollout an NCoverExplorer update.

Note - if you have downloaded NCoverExplorer or TestDriven.Net over the last week or so you may want to do so again. I have been sneaking in fixes for issues right up until this blog posting. Check the downloads page to make sure you have the very latest build number.

This is predominantly a bug-fix release, addressing a range of issues that have accumulated over the last six months. Of utmost importance was improving the coverage result accuracy from either the merging of multiple coverage.xml files or multiple AppDomains within a single file. Some of you have experienced issues with the "optimisations" of removing noop sequence points that NCover 1.5.4, 1.5.5 and 1.5.7 apply in different ways. This resulted in duplicated nodes in the treeview, or lower than expected coverage with brackets being shown as not hit etc. My thanks to everyone who contributed to the testing of this with their coverage files/repro projects and patience while I got the kinks out.

On the new feature front, the biggest is a "Find Type" dialog (ctrl+F) allowing you to navigate to types by entering their name (supports * for a wildcard).

A change to previous behaviour is to the fail on minimum coverage option for NCoverExplorer.Console. Based on feedback this has now been changed so that any assembly not reaching the threshold will cause the build to fail, rather than total coverage.

All the other tweaks and fixes can be found in the release notes.

I have updated the NCoverExplorer Extras download containing the NAnt and MSBuild tasks. The <ncover> task now uses a reference counting style approach to the registration of coverlib.dll, which should get around issues reported on build servers of simultaneous build issues if you got unlucky with the timings.

The task now includes a registerProfiler optional attribute for those of you using TypeMock - you can set this to "false" to ensure the NCover task does not try to register CoverLib.dll itself.

There is an enhancement to the <nunitproject> task to support the appbase property for people using multiple testing folders (thanks to Garth Williams).

Another addition is a new <ndoc2> NAnt task in there for anyone using NDoc2 Alpha as I blogged about in this post. Documentation is available via the links on this page but anyone familiar with the <ndoc> task should be happy. I've used it for generating CHM documentation without any problems.

For any issues with the release or further enhancement suggestions, please post to the NCoverExplorer forums or send me an e-mail (Help -> Send Feedback).

A quick further mention on NCover 1.5.7. This is a mandatory upgrade for every NCover user as far as I am concerned. Peter Waldschmidt's major rewrite of the profiler core and the addition of an integration test suite sets up a great platform for future releases. I've been fortunate to be able to chip in here and there on this release, although arguably more nuisance than help at times to Peter I'm sure!

I am highly optimistic however that this is the most solid NCover beta to date and was worth the wait to get it right. Top stuff Peter and many thanks for all the effort!

Last but certainly not least, Jamie Cansdale has made available a new build of his excellent TestDriven.Net add-in. Amongst other goodies this now includes the latest build of NCoverExplorer, NCover 1.5.7 and Visual Studio Express SKU support. Right-clicky clicky goodness...

Downloads
Release Notes

Filed in: ncoverexplorer ncover ndoc

Testing With NCover/NCoverExplorer

Jamie Cansdale — Sat, 11 Nov 2006 08:29:54 GMT

The following screencast demonstrates how to achieve 100% coverage using TestDriven.NET 2.0, NCover and NCoverExplorer. Peter Waldschmidt and Grant Drake very kindly agreed to let me distribute NCover and NCoverExplorer with the RTM version of TestDriven.NET 2.0. This means that once TestDriven.NET is installed, there is nothing more to configure. Code coverage with NCover can be used from Visual Studio .NET 2002, 2003 or 2005. It is possible to use NCover if only .NET 1.1 is installed, however .NET 2.0 is recommended. If only .NET 1.1 is installed then NCover 1.3.3 will be used rather than the more mature NCover 1.5.6.

In the screencast I start off by showing how to zoom in on the partially covered regions of code. I then navigate back to Visual Studio and expand my unit test to cover all of the partially covered method. Rather than create a unit test for the ToString method, I decide to use the 'Coverage Exclude' feature of NCover 1.5 (TestDriven.NET uses "//ea CoverageExcludeAttribute" by default). I create a new CoverageExcludeAttribute and use it on the ToString method (extra assembly references are required). This enables me to achieve 100% coverage with the excluded code grayed out and not counted in the final coverage score.

You can watch the screencast here.

I'd like to thank Grant Drake in particular for his support, generosity and convincing me that it was worth carrying on with the project. :-)

CoverageExclude

Jamie Cansdale — Wed, 04 Oct 2006 13:47:46 GMT

Eric Sink recently wrote an article, advocating the use of code coverage. In the article Eric writes about a hobby project of his where he has managed to achieve 100% code coverage. The following may be of interest to anyone who finds themselves driven towards such ridiculous levels of coverage. ;)

When using NCover, particular methods and classes can be marked with an attribute so they are excluded from the coverage report. TestDriven.NET tell NCover to use an attribute called 'CoverageExcludeAttribute' as this special attribute. When using the NCover console application, this option is:

//ea CoverageExcludeAttribute

The 'CoverageExclude' attribute isn't defined in an external assembly. This is because 'CoverageExclude' is most likely to be used in non-test code. We don't want to create a dependency on another assembly (just as you wouldn't want to reference 'nunit.framework' from non-test code). Instead the 'ExcludeAttribute' can be defined in the empty namespace as follows:

class CoverageExcludeAttribute : System.Attribute { }

Once defined this attribute can be used to exclude specific methods and classes from the NCoverExplorer coverage report.

.Net 2.0 TreeView Performance is killing NCoverExplorer

Jamie Cansdale — Tue, 03 Oct 2006 10:37:30 GMT

If you are familiar with the Windows Forms TreeView control, please take a look at Grant Drake's post on performance issues under .NET 2.0. You will find example code and a graph comparing its performance under .NET 1.1 and .NET 2.0.

It is upsetting to see the otherwise snappy NCoverExplorer turn into treacle when only .NET 2.0 is loaded (.NET 1.1 has precedence in the 'App.config' file). If you can find a solution, please let Grant know!

NCoverExplorer... v1.3.4

Jamie Cansdale — Tue, 18 Jul 2006 06:57:00 GMT

Syndicated from Kiwidude's Geek Spot

It's been a little while since the 1.3.3 release, but NCoverExplorer overload meant I had to take a break to keep some sanity. This release combines some important bugfixes with a small number of new features.

Firstly the new features:

On the toolbar of the NCoverExplorer GUI you will find four new buttons for navigating unvisited lines and classes. These have shortcut keys of 'N'/'P' (or forward/back mouse buttons) for the next/previous unvisited line in the current class. Use 'Ctrl+N'/'Ctrl+P' for next/previous unvisited class in this namespace (or Ctrl + forward/back mouse buttons). Thanks Jamie for the mouse button suggestion, it rocks!

You can use NCoverExplorer.Console.exe to merge and save your NCover coverage.xml files. Coverage exclusions will be applied prior to the save of the consolidated xml file. Wildcards are supported for the coverage filenames, so for instance you can do this:
```
ncoverexplorer.console.exe *.ncover.xml /s:coverage.merged.xml
```
If using the NAnt or MSBuild tasks from the latest NCoverExplorer.Extras.zip, you can add a new attribute "mergeFileName" (NAnt) or "MergeFileName" (MSBuild) containing the filename for the results.
Module level coverage thresholds are now possible using NCoverExplorer.Console.exe, rather than just a single threshold for your project. This is for people wanting more granular control where specific assemblies are deemed unrealistic to attain the same coverage as your overall goal, without seeing "failure" forever on coverage reports.
For NAnt, add this section within your ncoverexplorer task:
```
<ncoverexplorer ...
    <moduleThresholds>
        <moduleThreshold moduleName="MyAssembly1.dll" satisfactoryCoverage="30" />
        <moduleThreshold moduleName="MyAssembly2.dll" satisfactoryCoverage="80" />
    </moduleThresholds>
<ncoverexplorer>
```
For MSBuild, you can add this attribute to the NCoverExplorer task:
```
<NCoverExplorer ...
     ModuleThresholds="MyAssembly1.dll=30;MyAssembly1.dll=80" />
```
For complete examples refer to "example.build" and "example.proj" within the NCoverExplorer.Extras.zip file. If you do not want to use the NAnt/MSBuild tasks then you can use the command line with the /c option for a config file. The syntax is the same as for the NAnt ncoverexplorer task above - see the ConsoleExample.config file in your NCoverExplorer folder.
There is a new code coverage report, which drills down to classes per namespace per module. This is my "favourite" report we use on our CruiseControl.Net build servers. To generate this report the magic number is "4" for your report type attribute in the command line, NAnt and MSBuild tasks.

Updates to the CruiseControl.Net build summary stylesheet, so that it lists all of the assemblies, their coverage and whether they passed/failed as well as the overall project coverage. Note that this stylesheet now has a dependency on a transparent.gif file being placed in your CruiseControl.Net images folder. See the readme in NCoverExplorer.Extras.zip for details.

Added the ability to change source code paths for a loaded coverage file in the NCoverExplorer GUI. Sometimes you will be generating coverage files on another machine (such as a build server) that you want to browse - this feature allows you to override the embedded folder location to find the source code files in.

In terms of bug fixes, the most important that I blogged about recently is a solution for an issue when merging multiple coverage files. There are also fixes for memory leaks and various other minor GUI issues - see the release notes for full details.

As always, please drop me an e-mail through Help->Send Feedback if you have found any bugs or suggestions.

One final point - no doubt most people reading this have heard about Jamie Cansdale "going commercial" with TestDriven.Net. Jamie has been out there upgrading and supporting this great add-in for many years, for which many of us have enjoyed the benefits of "for free". Having had a small taste of what is involved in the last six months with NCoverExplorer you quickly gain an appreciation of just how much of your "life" supporting and enhancing these tools can consume! I wish him all the best in his opportunity to gain a return for those efforts - his continual innovation with TestDriven.Net continues to help fill the many gaps in the Microsoft "out of the box" tools to make for a far more productive development experience.

Download TestDriven.Net 2.0 combined install
Download NCoverExplorer 1.3.4 binaries

Download NCoverExplorer.Extras.zip (CC.Net/NAnt/MSBuild)

Release Notes
FAQ

NCoverExplorer... v1.3.3

Jamie Cansdale — Wed, 12 Apr 2006 17:37:00 GMT

Syndicated from Kiwidude's Geek Spot

Another stack of new features in the latest generation of NCoverExplorer. I've added summary html/xml reporting, filtering capabilities, basic code metrics and very significantly a console version for automated build usage.

Html reporting as I posted about here has been requested by a number of users, including myself! For this release I have implemented summary reports since from the feedback I received they were of most interest. From either the GUI or the the console application you can now generate one of three reports:

Module Summary
Namespace Summary
Module and Namespace Summary

As you can see from this dialog you have a choice of output either directly to html or to xml (the latter having an associated stylesheet for rendering html).

An example of the module summary report is shown here. Note the inclusion of some basic statistics at the top right. You can also see that our "satisfactory coverage threshold" is displayed at the top (90%) and that any graph bars exceeding this threshold (but not 100%) have the uncovered code shown in yellow. So a glance at the report for the red bars will tell you which modules or namespaces are possibly in need of investigation.

Statistics are also available within the NCoverExplorer GUI. Note that these are derived from the NCover coverage report, not the source code. The totals deliberately ignore any "excluded" code.

Filtering offers a way of simplifying down the coverage tree without affecting the coverage totals (unlike "excluding" which recalculates the totals). There are two filters available - hiding all 100% covered nodes (for when you want to focus on the todo list), and hiding all 0%/unvisited nodes (for when unit testing isolated functionality).

My thanks to Lutz Roeder for the public donation of his excellent CommandBar code. You can see the impact in this in the new look to the menus with images and the new Toolbar shown below.

Including nodes that were marked as "excluded" is also now possible. You can think of this as an "undo" feature to override either manual exclusions or those defined in the options dialog. Including a node will also recalculate the related coverage statistics.

NCoverExplorer.Console came about from two features on my list. One of these from Matt Hargett of imeem.com was to enable failing a build if coverage does not meet a particular threshold.

The second item was the ability to produce xml and html coverage summary reports in an automated fashion. As anyone who has used NCover on large projects will testify, coverage.xml files can be very large. Rendering these using xsl while being an elegant and flexible solution is just way too slow, particularly when the coverage file is many megabytes. Merging the coverage.xml file(s) into your CruiseControl.Net buildlog.xml files also slows down the rendering of every single page for that build on the CCNet web server.

My solution to this is based on the assumption that I do not want to see line level coverage information on the build server. I just want to see a summary of the project, modules and perhaps down to namespace level of coverage. By adding NCoverExplorer.Console.exe to the build pipeline we can transform the NCover results very quickly into a tiny summary xml file. With the appropriate stylesheets we can now render coverage summary reports in CCNet in a fraction of the time previously taken.

All of the options available in the GUI in terms of exclusions, thresholds, project name, output as xml and/or html are available.

I have also developed an additional stylesheet for use on the Build Report page of CCNet as shown below:

Since these are just stylesheets transforming the NCoverExplorer xml file they can be customised to your own tastes if required.

The final piece of the puzzle is to automate through NAnt and MSBuild the creation of the xml/html reports. You can of course just use an <exec> task, however I have put together some custom tasks for both tools which offer a bit more flexibility. These are available for download in the NCoverExplorer.Extras zip file at the bottom of this post along with some included examples and the necessary stylesheets for CruiseControl.Net.

If anyone requires it I may put up a blog entry walking through such configuration or perhaps see about getting it documented on the CruiseControl.Net wiki. It is fairly straightforward though:

Copy the NCoverExplorer.xsl and NCoverExplorerSummary.xsl stylsheets from the NCoverExplorer.Extras zip file to your CruiseControl.Net\webdashboard\xsl folder
Edit your CruiseControl.Net\webdashboard\dashboard.config file to point to these stylesheets. I have included an example dashboard.config in the .zip
Modify your NAnt or MSBuild script. As stated you can either execute the console application directly with command line arguments or use my custom tasks. For instance to use the custom NAnt task copy NAnt.NCoverExplorer.Tasks.dll to your NAnt\bin\Tasks\net folder and follow the example syntax contained within "example.build".
Modify your CruiseControl.Net project file (ccnet.config) to use the merge publisher to include the xml output from NCoverExplorer.

For my own sanity, other than any bug-fixes I am intending to put a hold on further development for a little while. This has consumed every evening and weekend for the last couple of months so it's time for a break. Any donations could of course help rekindle my interest ;-)

As always I welcome any feedback. Enjoy!

Download TestDriven.Net 2.0.1578 combined install
Download NCoverExplorer 1.3.3 binaries
Mirror Site

Download NCoverExplorer.Extras.zip (CC.Net/NAnt/MSBuild)

Release Notes
FAQ

NCoverExplorer... v1.3.2

Jamie Cansdale — Wed, 15 Mar 2006 11:35:00 GMT

Syndicated from Kiwidude's Geek Spot

Plenty of "must have" features that have been on people's wish lists in this release and a couple of irritating bugs fixed so I can't recommend highly enough that people upgrade. Download details at the bottom of the post.

Merging coverage files is the first of the "killer" features, offering improved integration with TestDriven.Net. Some of you may have noticed the inability to produce coverage results across a VS.Net solution or with multiple fixtures selected in the solution explorer window. The latest release of TestDriven.Net now supplies multiple coverage file names to NCoverExplorer which will merge and present the consolidated results. You can also manually merge multiple coverage files within NCoverExplorer via a number of ways, such as a "File->Merge" menu option or drag/drop.

Saving of the consolidated coverage results is now possible using the new "File->Save/Save As" options. Note that saving will remove any excluded nodes (see below) and merges together any duplicated visit count sections from the new coverage.xml file. This means you will likely find the resulting file much smaller than the originals.

Coverage exclusions are my personal favourite new feature. This offers a way for NCoverExplorer to automatically remove from the coverage tree items not relevant to your analysis when a coverage file is loaded - such as unit test fixtures, third party libraries etc. In the screenshot shown you can see the default exclusions I have included with NCoverExplorer - for instance any assemblies ending in ".Tests" will be excluded. You can add exclusions at the class, assembly and namespace level. I have also added support for the NCover 1.5.x feature of "exclusion attributes" if they exist in the coverage.xml file (although NCover still has some kinks to be worked out in a future release before you can rely on this feature).

All excluded items are then added to a special Excluded bin for the parent node as shown in the screenshot below. The source code for excluded methods will be highlighted in grey. You can also manually exclude items on the fly via a right-click "Exclude from Results" menu which replaces the previous "Remove from Results" option.

Source code tabs started out as a solution to a problem a few users reported with highlighting, particularly to do with partial classes. I believe I have resolved this once and for all now with the introduction of this feature. If you click on a partial class in the coverage tree, a tab is opened for every source code file making up that class with all the code highlighted as shown here.

The tabs also offer a way of quickly navigating around the coverage tree as selecting a tab will select the corresponding class node.

Themes came about as an answer to another much requested feature of more customisation of the GUI to include fonts and background colours. A "theme" is a group of settings covering most aspects of the appearance of the coverage tree, statistics pane and source code highlighting.

You can of course add your own themes as shown in the screenshots on this blog entry. Themes are easily toggled between using the new View->Themes menu.

Sorting and Coverage options came from a couple of user feature requests as shown here.

These allow NCoverExplorer to display function coverage or focus on methods with the most unvisited code.

My thanks to Trevor Hall for his time taken to suggest and test not only these features but also a number of others slated for a future release (after I take a break!).

Thanks also to Jamie and others who have been kind enough to test the beta versions and provide invaluable feedback.

There are numerous other minor usability tweaks and enhancements I haven't mentioned here that you can read about in the release notes or discover for yourselves. I look forward to any feedback!

Download TestDriven.Net 2.0.1545 Combined Install

Download NCoverExplorer 1.3.2 binaries
Mirror Site

Release Notes
FAQ

NCoverExplorer... v1.3.1

Jamie Cansdale — Thu, 16 Feb 2006 21:24:00 GMT

Syndicated from Kiwidude's Geek Spot

A ton of new features in the latest release, and a few bugs fixed ranging from minor annoyance (code highlighting) to downright critical for some (supporting .Net 2.0 features like partial classes and generics - thanks to Jamie Cansdale & Andreas Hofer in particular for pointing out the issues!).

As per usual you can download this version with the just released build (2.0.1435) of TestDriven.Net or as a standalone product via the links at the bottom of this post.

What's New? Loads... more than I'm going to mention here (see the release notes for the full list). Let's look at the options dialog:

Apart from the new tabbed appearance there are some other new options to note here:

Namespaces will now be "flattened" by default which is the same appearance as Reflector or the VS.Net 2005 class viewer (see screenshot below).
For the VB.Net crowd out there you can now exclude the "My" namespace by default which NCover includes in the results, distorting your coverage percentages.
An optional threshold for the number of unvisited lines in a method. Acts in a similar way to the existing % threshold feature but just another way of convincing yourself things aren't as bad as they may first seem...

Probably the most requested feature I have had is the ability to customise the colours - so here you go! There is more to come in the future on this but for now you can change:

Moving onto the display itself, you can see some new menu options in the tree view and the default view of a "Flat" namespace hierarchy below:

The two new right-click menu options are:

Remove from results - does what it says on the tin. This option is context aware so you can use it to "prune" your coverage tree - be it to remove a unit test assembly as shown above or exclude a class. After the remove action is selected NCoverExplorer recalculates the percentages in the tree.
Collapse All - collapses every single coverage node in the tree.

Next is the statistics pane. This has had quite a revamp (my thanks to Andrzej Kaczmarczyk for his suggestions). Clicking on a class node now produces a summary of all the methods and the number of times they were executed (shown below). The columns are now sortable as well. This offers a rudimentary form of code profiling analysis.

Clicking on a method node is also slightly tweaked to show the method name.

The final new feature I think worth highlighting is that by default NCoverExplorer now tries to restore your cursor position / current selected node when reloading a coverage file. So when used in conjunction with the superb TestDriven.Net "Test With... Coverage" feature you no longer need to keep re-expanding the nodes to get to where you last were...

I look forward to your feedback with any additional suggestions, comments... or even dare I suggest new bugs! This release has made quite a dent in the features todo list but there's still a number of things on it yet...

Also very much worth noting - the latest NCover beta release 1.5.3 is now available - highly recommended for the bug fixes alone. Thanks Peter!

Download NCoverExplorer 1.3.1 binaries (203KB)
Mirror Site

Release Notes
FAQ