Lack of Coverage Tools for .NET

For the longest time, I've been using NCover for coverage. It was free, gave good results and with NCoverExplorer and CI integration it was the perfect tool. Easy to use, easy to setup, and worth the typical hassle of new tools (setup, learning, configuration, etc.)

NCover has since gone commercial and the old versions won't run properly against 3.0/3.5 code. I'm ditching NCover support in TreeSurgeon because of this. TS nees to be friction free and something anyone can just grab and run and not have to deal with buying or obtaining licenses for products the generated code depends on. I looked at Clover.NET as an alternative (last time I checked it was free?) but it's $600 just for an "academic" version.

So what's with coverage and .NET these days? Are these the only options? Have all the open source/free tools gone the way of the Dodo for coverage and .NET 3.0 projects? My quick Google checks this freezing morning (it was minus 38 as I drove in this morning, that's -36 in Fahrenheit kids) don't show anything.

To be friction-free a code coverage tool needs to be:

  • Command line driven (have to be able to integrate into continuous integration process)
  • Xml report output (again, CI ready)
  • Usable with any unit testing framework
  • Support 3.0/3.5 .NET framework
  • Somewhat stable (i.e. non-alpha/beta versions)

Nice to haves:

  • Free (or relatively cheap)
  • HTML or Text output
  • Ability to set thresholds and get failures back (for example, break the build if the coverage falls below 80%)

Looking for some feedback on your experience here. Thanks!

Published Tuesday, January 29, 2008 7:01 AM by Bil Simser
Filed under: , ,

Comments

Tuesday, January 29, 2008 9:42 AM by peter ritchie

# re: Lack of Coverage Tools for .NET

Maybe someone needs to start another OSS project for code coverage?  I'd be willing to pitch in...

Tuesday, January 29, 2008 9:42 AM by Sean Chambers

# re: Lack of Coverage Tools for .NET

You make a very good point sir. This has definately been a pain point for me. Especially since ncover was sucha  great tool. How much are they actually charging for the commercial version?

Tuesday, January 29, 2008 10:51 AM by Bil Simser

# re: Lack of Coverage Tools for .NET

@Sean: NCover was a great tool but now it's useless with 3.0 code projects (unless you buy the new version which I assume works with 3.0). Cost isn't huge (compared to Clover) and a copy will run you $150USD. However I consider that pretty pricey for the features you get back. Compare it with something like R# which is about the same price but has 10x more functionality. A code coverage tool is just that and should run in the $50 price range.

Tuesday, January 29, 2008 11:43 AM by sodablue

# re: Lack of Coverage Tools for .NET

This is generally one of the problems I have with so many developer tools.  I understand that they have a somewhat limited audience, but anything over $50 puts it outside of my pain threshhold.

I've bought things like ultraedit, beyond compare, filesync and such, because they do something for me and the price point of $25-30 makes it a "No duh" purchase.

But when someone is asking $150 or more, it's a struggle to justify, especially given most developer tools only last a year or so before the underlying technology changes beneath you and you have to buy a new copy and in most cases that appears to mean a very limited discount.

It'd be one thing if my company would buy us tools, but we've got no formal process for this.  I spent six months trying to fight for a copy of redgate's profiler and was never able to get it.

Tuesday, January 29, 2008 11:48 AM by Eden Ridgway

# re: Lack of Coverage Tools for .NET

You might want to look at PartCover (sourceforge.net/.../partcover). It seems to be a pretty good free substitute for NCover.

Tuesday, January 29, 2008 12:09 PM by Mario A Chavez

# re: Lack of Coverage Tools for .NET

Project Mono has monocov as a coverage tool, but is some how tied to mono, maybe it can be used as a start, here is the code:

anonsvn.mono-project.com/.../monocov

Tuesday, January 29, 2008 12:47 PM by Pavel Samokha

# re: Lack of Coverage Tools for .NET

SharpDevelop team recently switched to PartCover - AFAIU currently this tool doesn't have GUI, so it may be good to cooperate with SharpDevelop team to create OSS GUI tool.

laputa.sharpdevelop.net/CodeCoverageAddinUsesPartCover22InsteadOfNCoverInSharpDevelop3.aspx

Tuesday, January 29, 2008 1:55 PM by Chad

# re: Lack of Coverage Tools for .NET

I think the problem is that a.) code coverage is so hard in .NET due to the crazy COM stupidity hoops you have to go through to do anything useful b.) now that automated unit testing is gaining adoption, PHB's eager to gauge the productivity of their developers have turned from LOC counting to Coverage counting (or both -- with tools like NDepend and Clover.NET/NCover).

I'm not saying NDepend and Clover/NCover are bad, quite the contrary, but I think there's real money to be made by people who are trying to apply command and control-style management to unit testing.

I'm also not saying that is bad, but it leads to a lucrative market with a steep barrier to entry (due to the COM craziness involved in making a CLR profiler/code-coverage analyzer).

Tuesday, January 29, 2008 2:41 PM by Tom Opgenorth

# re: Lack of Coverage Tools for .NET

To jump on the band wagon, I've been meaning to look at PartCover for my current contract.  If you get there before me, let me know.  :)

Wednesday, January 30, 2008 10:20 AM by Fabrice Marguerie

# re: Lack of Coverage Tools for .NET

You can see if this list if there are code coverage tools for .NET that you don't know: sharptoolbox.com/.../code-coverage