lowering the adoption barrier for unit testing: doctest in python

One area that I didn’t know about was brought to me by an attendee in one of the conferences I was in. It’s a simple idea, really: write the tests for a method as part of the method’s documentation. but make them executable.

In python, they have something called doctest (thanks to Alf Lervåg for the emails!)

here’s a PDF with more in.

 

With the upcoming WPF editor in VS 2010, we could find a good use for such a thing. I think it might ease adoption of unit testing becuase it’s much easier to accomplish if you can write the test right there where the method you’re writing is. The cons are lack of intellisense, compilation errors and such, but it’s something to look at, indeed.

Published Wednesday, December 03, 2008 5:18 PM by RoyOsherove
Filed under: ,

Comments

Wednesday, December 03, 2008 8:04 PM by Ryan Riley

# re: lowering the adoption barrier for unit testing: doctest in python

That would indeed be outstanding. I often place code examples in my comments, and these could easily become unit tests. In fact, many of my unit tests often resemble the examples in my comments.

One further thought: with IronPython in the mix, couldn't we use doctest with C#?

Wednesday, December 03, 2008 11:12 PM by Judah Himango

# re: lowering the adoption barrier for unit testing: doctest in python

Wow, that's a great idea! But yeah, lack of intellisense and all that nice IDE magic would be a real downer. It would also seem the tests you can generate are rather limited -- no clear way to isolate dependencies using only "documentation tests".

Interesting idea. If someone takes up the project, I'd be interested in contributing.

Wednesday, December 03, 2008 11:52 PM by Mark Roddy

# re: lowering the adoption barrier for unit testing: doctest in python

My experience with the doctest module is that it seems to scale poorly in regards to testing object oriented code, as you don't have the support of the underlying testing library to assist in writing the tests.  

I get the argument that doctests lower the barrier to entry by eliminating the need to learn a testing library, but if you're using python already you have the unittest module as part of the standard library which is one of the simpliest varients of an xUnit library in my opinion.  

I'm be leary of recommending doctest to people out of fear that they'd get frustrated after a few weeks without hard it is to write tests that take more then 3-4 lines of code that would be much easier with unittest.  Of course this comes down to the ease of entry versus ease of use argument I've been seeing in numerous blogs dedeciated to testing.

-Mark

Wednesday, December 03, 2008 11:54 PM by Mark Roddy

# re: lowering the adoption barrier for unit testing: doctest in python

My experience with the doctest module is that it seems to scale poorly in regards to testing object oriented code, as you don't have the support of the underlying testing library to assist in writing the tests.  

I get the argument that doctests lower the barrier to entry by eliminating the need to learn a testing library, but if you're using python already you have the unittest module as part of the standard library which is one of the simpliest varients of an xUnit library in my opinion.  

I'm be leary of recommending doctest to people out of fear that they'd get frustrated after a few weeks without hard it is to write tests that take more then 3-4 lines of code that would be much easier with unittest.  Of course this comes down to the ease of entry versus ease of use argument I've been seeing in numerous blogs dedeciated to testing.

-Mark

Thursday, December 04, 2008 1:21 PM by Scott Bellware

# re: lowering the adoption barrier for unit testing: doctest in python

If we're going to go this route, Cucumber in Ruby is possibly quite a bit better as an option due to maturity and adoption.

Tuesday, December 09, 2008 6:40 PM by John P. Brice

# re: lowering the adoption barrier for unit testing: doctest in python

I think the bigger problem with adoption is getting people to see the value in it.  Honestly, even if the doctest module read their minds and all they had to do was 'think' about testing the code and coming up with test cases they'd reject the idea.

Recently I gave a presentation and I quoted from your book the bit on the case study of the project that was done with and without unit tests side by side, and still even after pointing out in that class they weren't ever going to see the value in unit testing if they can't see the 'forest' many people still, immediately after I finished that section and pointed out the overall *less* time, and 1/10th the bugs, went back and immediately said 'but it took twice as long to code'.   It's like they're in an infitie loop:

while(happilyEmployed && fortyHourEmployee)

{

   if(futureCodeTime > currentCodeTime)

   {

       FullyRejectAndDisqualify(idea);

   }

}

I think back to prior to my 'aha' moment and can't just recall what exactly it was that made me 'aha' or why I was so hell bent on unit testing being a bad idea.   That's what we need to lower the adoption barrier - put that 'aha' item in a 'can' and go from there.