Extreme JS

XP StoryStudio v0.99b released

jsgreenwood — Tue, 18 Jan 2005 21:42:00 GMT

Having gathered initial feedback from the v0.99 release, v0.99b is now available, with the following minor improvements:

Iterations can be deleted. In v0.99, this feature just linked to the Story Delete, and wasn't implemented
Date fields now have pop-up calendars
Installer now supports choice of whether to run/not run tests following installation
Installer now supports not configuring the database
Installer will now work without SQL Server being installed on the local machine

v0.99 is now no longer available for download. However, v0.99b is available from www.xpstorystudio.com, or directly by clicking here.

To perform an upgrade from v0.99, first uninstall the existing version (this will NOT remove your existing database), then run the v0.99b MSI installer and select the database upgrade version.

Active and passive risk

jsgreenwood — Sat, 08 Jan 2005 19:05:00 GMT

Attitudes towards risk are something that I've been meaning to write about for some time now. To me, there're basically two types of risk - active and passive (I'm no risk expert in the "risk analyst"/academic sense, so ignore all of this if it's obvious)...

Active risk is that which is deliberately taken on - for instance the choice to develop a new product that may (in theory) fail in the market. Or the rewrite of a piece of software due to burgeoning support costs.
Passive risk is that which is inherent in inaction - for instance, the choice not to update an existing product to compete with others in the marketplace. Or the decision not to rewrite a piece of software, despite burgeoning support costs.

Both these types of risk can be measured in the same way - the cost, and the potential return/loss. Yet people seem to have very different attitudes towards them. Passive risk is seen as a necessary evil that's often ignored. Whereas active risk is seen as something to be avoided, regardless of the potential payback (and likelihood thereof).

I think, in corporate life, the problem lies in what people are measured/judged on - the decisions that they DO make (active risk), rather than the ones they DON'T make (passive risk). It's easier to blame indecision on someone else than it is a choice you made yourself. Unfortunately, I've seen many cases where the passive risk is huge; easily enough to cause a company to go under sometimes (and actually causing it to, in at least one company I've worked with).

In theory, this all comes down to a corporate risk register, and ensuring that it's both complete, and has well-defined accountabilities for each item. Unfortunately, I've rarely seen this really working, with numerous passive risk items dropping out of sight due to an unwillingness to take on responsibility.

Agile planning tool - XP StoryStudio - available for download

jsgreenwood — Tue, 04 Jan 2005 00:49:00 GMT

Whilst I've not had time to do all the documentation I was planning to over Christmas, I have got round to bundling up the installer and updating www.XPStoryStudio.com.

So, available for download now (from the aforementioned URL), is v0.99 of XP StoryStudio - the freely distributable XP planning tool created by myself and Dave Fellows, with the help and support of several other kind souls at Egg.

From the documentation enclosed within the installer:

This is a functionally complete, pre-release version of XP StoryStudio
intended for limited distribution to test installation and integration issues
prior to the public release. As such, should issues be found, please forward them to: support {at} xpstorystudio.com.

When TDD Goes Bad #2

jsgreenwood — Wed, 29 Dec 2004 22:35:00 GMT

I've been debating whether to post another entry on this for the last couple of weeks due to another bad smell I've spotted around Agile practices. However, it's my blog, and I think it's worth saying. But first, the bad smell...

Zealotry. Many advocates of Agile practices have had to fight their corner hard against supporters of traditional methodologies. Having had several such confrontations myself, it does become tiring, and you do beging to expect fairly incongruent arguments against it. There is a danger, though, that the habit of dealing with arguments against Agile that don't stack up will lead to a blinkered view where it's assumed that any negative statement about the principles and approach should be dismissed. No methodology is perfect - different circumstances call for different approaches, etc. Also, just because a methodology when applied correctly is appropriate and should result in success, it doesn't mean that it will be applied correctly and appropriately. People can, and will, get things wrong - especially when lacking experience.

So, TDD Gone Bad #2... Mock oriented development. Mocks are a good thing where you really want to extract a dependency and ensure you're testing the right thing. But there are a couple of issues that can arise:

Poor integration tests, as everything is being tested in isolation - we can end up with a system where the constituent parts are clean, isolated, well tested, and known to be correct. But how they fit together is a greyer (or even blacker) area unless mocking is accompanied with a complement of integration tests. Dave describes this well in his blog, so there's no need for me to go on about it here: http://www.twelve71.org/blogs/dave/archives/000616.html
Mock oriented design. Whilst mocking is good at removing dependencies, their introduction regularly alters the design of the system. I've seen numerous occasions where the introduction of mocks has added a large amount of complexity to an otherwise simple design. This complexity leads to higher implementation costs, a higher cognitive load on the developers working on the system, and higher maintenance costs (as there's more code to maintain). All of which go against the principle of "the simplest thing". The irony is that the introduction of mocking can, sometimes, make completing a system far more time consuming due to the different levels of granularity, and the additional code required to implement interfaces, etc. Again, that's not to say mocks aren't very useful things, just that they're a tool to be used where appropriate, not a pattern to base the foundations of your entire implementation on...

Whenever mocking is used, the value that the mock gives versus the cost it will introduce over the lifetime of the system should be measured (or at least estimated/considered)

When TDD Goes Bad #1.2

jsgreenwood — Wed, 08 Dec 2004 16:06:00 GMT

So, #1.1 was all about the "business" - people defining requirements, and how these can cause issues. #1.2 is just a short entry about the underlying statement I was trying to make in the original post:

Anti-pattern: Where inexperienced/misguided developers keep on testing where it's not realistic or financially astute to do so, thinking that's what TDD is, and thinking they're "adding value". As we all know, that's not what TDD is. And it's something we should try to avoid by coaching, mentoring, and working closely with development teams; helping to give them ways of judging the value of any piece of work.

It's all too easy to work with a group of savvy technical people that get TDD, and not be able to see how it could go wrong. If you try and scale this across an enterprise where some people just don't get it, and the support isn't there to keep things moving in the right direction, things can, and do go wrong.

When TDD Goes Bad #1.1

jsgreenwood — Tue, 07 Dec 2004 17:43:00 GMT

OK, so people don't seem to get what I was trying to say in my previous TDD post, to the point of claiming (on certain newsgroups) that I "don't get TDD". There were two points I was trying to make. So, I'll try and lay out the first of these now, in a simpler manner:

Every test you write has a cost associated with it - the development time/cost: C
Every test you write has a value associated with it - the time/money it'll save over the lifetime of the system: V
At its simplest level, V must be greater than or equal to C for it to have been worth implementing the test (I'll probably post something separately on the mathematical modelling of this in the future)

The reason that V isn't always greater than C can arise from several situations from a "story" (business) perspective:

Developers have been provided with open ended requirements - where there is no bounding to the tests that have to be written. At its most extreme case, this could be something like "Test: An unexpected physical occurence takes place outside the system. Result: The data will be backed up correctly". That's an open ended test, though. What if someone cuts power? Or a lightning bolt hits the server rack? Or a tiny blackhole forms over the CPU? "C" would become infinite if all scenarios were dealt with (just because I've used an extreme example here doesn't mean it couldn't be something much more subtle, either). Some of these scenarios are more likely than others, and there's diminishing return on each test being implemented as the likelihood of the scenario occuring drops. To stop this situation from occuring, requirements need to be closed and finite. If they're not closed/finite, you arrive at the situation in my previous post where a developer can simply "judge" how much testing is enough. Unless developers are they calculating the C:V ratio of each test in a situation with any open-endedness, then TDD can go bad.
Developers have been provided with requirements with negative value - where the cost of implementing the test + code is inherently greater than the value it will deliver. Accurate estimation of tasks and comparison against the value of any given story will avoid this situation.

[In part #1.2, I'll discuss the second part of this]

XPStoryStudio site up: www.xpstorystudio.com

jsgreenwood — Sun, 28 Nov 2004 20:36:00 GMT

Having registered the site a couple of weeks ago, I've got round to putting together and uploading a couple of web-pages for XPStoryStudio, in preparation for it's release in the very near future. All that's left to do is put some installation documentation together and generate the finished installer package. In addition to posting major news about its release, etc. here, all the details will be published on XPStoryStudio's site:

www.xpstorystudio.com

When TDD Goes Bad #1

jsgreenwood — Fri, 26 Nov 2004 00:58:00 GMT

Although Test Driven Development (TDD) is one of the greatest steps forwards in software engineering, especially when combined with modern languages and testing frameworks (i.e. xUnit), there's a definite anti-pattern lurking in there - Test Oriented Development.

At its most basic level level, writing tests is done for one reason and one reason only. It's not about ensuring you meet requirements, it's not to have a repeatable means of knowing the code functions properly after changes, and it's not to aid design. The key reason for writing tests is the same as everything else in software development - to make the company commisioning the software more profitable. So, the justification for TDD is based upon the premise that, in the long run, writing tests will save the company money on down time, effort in re-workings, and so on - but these are just implementation details of the underlying principle. Now, although most developers that do TDD understand the long-term payback view of the approach, I'd question how many actually consider the value of each test or set of tests that are being developed.

Using an agile methodology, tying the (acceptance) tests to the stories is fairly simple, seemingly giving visibility of the value of tests. But, given that many verbal tests are quite open ended, how do you know how much testing is enough? The test may be stated as "When I enter the correct data, but there is a connectivity issue, I will see error message X". A verbally described test such as this can have a hundred different programmatic tests implemented - simulating standard firewall issues, XML-firewall issues, web servers being down, latency in connections, time-outs, corruption of data, etc. There is no immediately visible answer to "how much testing is enough?" in such a scenario. This is even worse with unit tests, where they aren't likely to be defined formally (verbally) at all, and are largely up to how meticulous a developer is.

Added to the lack of metric is the fact that the developer is concentrating on testing before "coding" - the primary function is hence the development of tests, leading to a situation of "test-oriented development". This is where the production of tests becomes the overriding concern and output of a team, with the development of required functionality being secondary. When this occurs, the design, quality, and scope of the tests all become greater than that of the system actaully being created. As the development of tests follows the law of diminishing returns principle, beyond a point, each additional test you write for a certain circumstance adds only more and more marginal increases in quality. Eventually, you will reach a point where there will never be a point in the lifetime of the system where the development of the tests achieves ROI. Basically, the development of tests becomes counter-productive to the aims of the business. This is compounded by the Heisinger's Uncertainty Principle-like facet of TDD - it's alteration of the design of the code it's testing. But that's a subject for another post in the next few days...

Avoiding test-oriented development can be quite difficult - changing a developer's mindset to take on TDD is difficult enough to start with, trying to then amend that to temper the creation of tests goes back on that principle to some extent. My current thinking on this is that, along with acceptance tests and development tasks, a separate list of requirements needs creating for each story - a list of required unit tests. Rather than just estimate the effort required to complete a story point in its entirety, the effort required for each unit test (and its value in pound-notes/dollar-bills) should also be calculated to some extent. Each test is then subject to the same cost-benefit analysis and prioritisation as stories at a higher level. The task of doing this should also help developers to consider the reason for creating a test; how realistic/contrived the situation it's testing is, how better the effort could be expended, and finally, help remind them what they're actually trying to achieve at a less programmatic level. If this seems like too much effort, then an alternative may be to look at the ratio of time that is to be spent on writing tests to that of the production code itself for a given story-point to ensure that the focus isn't shifting from the system to the tests.

Coordinating Enterprise Website Development in .NET

jsgreenwood — Wed, 10 Nov 2004 22:25:00 GMT

One of the favourite Enterprise development strategies for .NET websites I've come up with is that of splitting the website into logical areas along functional- (and naturally change-) boundaries; having separate areas of the site developed as separate ASP.NET controls. I'm not talking small-scale controls like "Address entry" here, I'm talking entire 5+ page application processes as a single control (that might itself comprise further controls). This model works well for all kinds of large organisations - an online e-commerce site could have separate basket, checkout-process, and product-search controls, for instance, each of which would be aligned against a set of services that they're consuming (in an SOA). The current enterprise-in-question is a bank, which fits this model better than most other sites I've come across due to the sheer amount of functionality necessary on the website, and its diversity. Application forms alone constitue a raft of controls:

Mortgage applications
Credit card applications
Personal loan applications
Motor insurance applications
Etc.

My view on coordinating this has been, for some time, to have separate teams working on each "control", basically becoming product teams that deliver a black-box control to one overall "site" team. It is then this team's responsibility for plugging all of the controls into the main site, maintaining all the links between areas, etc. There are several benefits to implementing an enterprise-class site this way:

It gives a simple way of logically breaking-down/grouping a large number of development staff
It supports concurrent development on multiple parts of the site. Because the controls don't know about each other, there's no risk of them linking to one another and dependencies being created. This allows for varying rates of change in different areas, and doesn't predicate large-scale upgrades when a new version of some base functionality becomes available
It provides a black-box component that allows for alternative internal implementations, for instance migrating from an HTML based UI to a Macromedia Flash one, and eventually to a Longhorn one.
It matches a model of having multiple levels of continuous integration, rather than having a single-level uber-integration that becomes unwieldy

That's the good news - a paradigm for developing enterprise websites that theoretically has few drawbacks. Moving on to the implementation is where the bad news begins...

In .NET, these controls could either be developed as user controls (.ascx files), or as custom controls (.cs files). The advantage of the former is the speed of development, and close integration with UI designers - developers can work in "HTML view". The latter involves writing all content as controls. A short example of this is as follows:

Web control implementation:
<h1>Hello World</h1>
<asp:TextBox id=txt runat=server />

Custom control implementation:
Literal literal = new Literal();
literal.Text = "<h1>Hello World</h1>";
Page.Controls.Add(literal);
TextBox txt = new TextBox();
txt.Id = "txt";
Page.Controls.Add(txt);

Although possible, developing a graphically compelling site using the second method is torturous, so I've discounted it as an option. I also don't want control teams to deliver an editable ASCX control to the site team - it would be prone to editing, etc. and not as much of a "boxed product". There is an alternative, however. Whenever .NET renders a web page that makes use of a user control, it converts the first code example into the second internally. In Visual Studio .NET 2002 and Visual Studio .NET 2003, this process is largely invisible, and only happens when the page is requested. Visual Studio .NET 2005 ("Whidbey") changes this, though, allowing for pre-compilation of websites into Assemblies (DLLs).

So, my long-term plan was to have each team develop a control in a separate VS.NET 2005 project, pre-compile the output when they're ready to release, and then just hand the Assembly over to the "site" team to use just like any other class library. Having spent many, many hours over the last week poking and prodding the aspnet_compiler.exe tool, rewriting it from scratch, attempting reflection on the System.Web.Compilation namespace, and even disassembling most of the System.Web assembly, I'm getting close to giving up. Rather than creating one sensibly named assembly for the whole project, the compiler creates multiple randomly named assemblies - usually one for each control/page (although it sometimes puts several together). After recreating the aspnet_compiler tool from scratch, and extending it to sift through these files, pull out just the assemblies, give them new file names based on the controls they contain, etc. I thought I was getting somewhere. Now the problem is that if one of these controls uses another from the same project, it will be given a reference to the random 8-letter assembly name (and matching internal module name). I've yet to see if any internal assembly hacking can rectify this.

None of this would be necessary if Microsoft hadn't specifically made the majority of the System.Web.Compilation namespace private/internal, presumably in an attempt to make it difficult for other people to develop competing tools, etc. The most infuriating part of this is that, in theory, you can access all of the System.* functionality through reflection, whether its supposedly private or publicly visible. Additionally, tools like Salamander and Reflector make it possible to decompile vast swathes of the system classes back into C#. So Microsoft aren't really "protecting" anything, they're just making it painful to come up with enterprise class solutions.

The point to this post?

Here's a model for enterprise web-site development that works, even if it DOES mean handing over controls that aren't totally black-boxed.
If anyone is a System.Web.Compilation guru and fancies helping out on creating a tool to enable this, please mail me
If anyone thinks that Microsoft should make it easier to compile source code, please mail them

SOA Design with Agile methodologies

jsgreenwood — Sun, 17 Oct 2004 19:56:00 GMT

Service Oriented Architecture (SOA) and Agile - two of the current "hot topics" in IT. The interesting thing is that even though they're fundamentally different - one being a set of architectural principles, the other a set of methodology principles, one of the key goals and benefits of both is enabling change.

Now, there are clearly differences in the goals (enterprise integration vs. fail early, etc.), but that's beyond the scope of this entry. What this is about is that these are two sets of principles that are both useful and appropriate in today's technology landscape. So, a natural situation to arrive at is where you want to transform a company to use both Agile and SOA. This raises an interesting issue, though...

Agile methodologies promote an incremental, iterative approach to development of functionality (including method signatures), with visibility of the impact of change given through test coverage. Basically, working on the premise that, change is cheap if supported correctly
SOA promotes a well defined service interface through contracts - these contracts are aligned against business processes, not implementation details. The rigidity of the service interface, and its non-technical alignment allows for internal change without impacting consumers of the service. Changing a service interface is an involved process, though; rather than an outright change, a versioning mechanism is needed due to the immutability of contracts, and therefore a migration process is required. The visibility of service usage is also very difficult to monitor as it could be external organisations rather than internally controlled systems. The upshot of these two factors is that change to the interfaces of a service are relatively expensive

So, whilst the internal development of services and service-consumers is in-line with Agile, the creation of service interfaces isn't - an incremental, iterative approach to those would cause serious versioning and compatibility problems, especially with remote (3rd party) consumers. Similarly, imagine you're on the receiving end of another company that has an Agile approach to a service interfaces you're consuming; the contract you're adhering to would shift under your feet continually, introducing additional cost.

Thankfully, there is not only a solution to this problem, but it is also one that actually benefits Agile methodologies. The solution is to make use of the incremental aspects of Agile, without the iterative approach when designing service interfaces. Basically, the service interfaces should be defined just-in-time; when the first requirement for a certain piece of functionality arises, the definitive implementation of the interface for that particular service method and no more is derived. The removal of an iterative approach to interface design is important not just for cost reasons - one of the key factors in the success of an SOA is how accurately the service interface mirrors the real world process. If deriving a clear definition of this interface cannot be done, then it generally means that the underlying process isn't well understood.

As for the benefits of this to Agile, it's long been my opinion that the largest problem with such approaches are the introspective facets of them - the virtual disregard for what isn't clearly defined and within the primary context. Service interfaces act as change boundaries for these introspective "pockets", containing them from unintentional impact on other such "pockets". This allows iterative approaches to be taken to the component- and object-oriented internals of services, without incurring the cost of repeatedly integrating between potentially external, non-visible systems.

User Agents - Tomorrow's Technology Today

jsgreenwood — Thu, 14 Oct 2004 19:58:00 GMT

For years, there's been talk of tomorrow's technology - user agents, and how they'll make our lives better. How we'd be able to sit there waiting for information to come to us, with agents going away collecting it, collating it, and presenting it back. How this would change our lives. Like many, I've been of the opinion "I'll believe it when I see it" for a long time, and it kind of got written off and pushed to the back of my mind. And then I realised, user agents are here and have been for quite some time. It's just that the reality's not quite how I (or I guess others) envisaged it to be.

From the very first e-mail client, we've had user agents; desktop applications that communicate with remote servers and bring down data ready for when we want it, and notify us of its arrival. The addition of out-of-office replies, then Rules Wizards, then Junk mail filters are all moving the likes of Outlook in the direction of the archetypal "user agent". But the presentation is still largely standard desktop client. Other examples of such applications are the MSN Messenger Hotmail notifier and RSS readers.

To me, the two true prerequisites for a real user agent world are:

Web services/XML - a common means of requesting, retrieveing, and interpreting data.
A client-side infrastructure for displaying this information

Whilst the first of these is now available, standardisation of schemas is currently one of the biggest problems. Even RSS, one of the most widely used standards, has several different versions (0.90, 0.91, 2.0), and competition from the likes of Atom. What the world really needs is standardised ways of representing everything else - personal information, stock quotes, and so on.

A larger problem exists in the client-side infrastructure. Support for platforms besides Windows aside, there is currently no simple way built into the platform of writing such an agent that "integrates" with the environment and other such agents whilst having adequate usability. If you look at all the agents so far - E-mail clients, RSS Readers, MSN Messenger, whilst there are attempts to integrate all of these things, it's neither controlled nor elegant. In the future this, problem will be solved with the Longhorn Sidebar, along with MSN-esque popups. In the meantime, the closest thing I've seen so far is www.desktopsidebar.com; a true framework for integrating various agents that borrows more than a few ideas from the Longhorn Sidebar. Supporting Messenger, performance monitors, weather feeds, RSS feeds, the obligatory stock quotes, and so on, as well as being a framework for hosting further agents and giving them a UI with which to interact with the user.

So, agent technologies are here, they just crept up on us quietly - evolving over time rather than arriving with a fanfare fitting the amount of hype around them.

Tool updates

jsgreenwood — Thu, 07 Oct 2004 22:20:00 GMT

Long time since my last post - I've been getting ready for and going on a nice bit of pan-European road racing. The car was getting a fair bit of stick, so didn't make it the full distance, but that's another one to cross off the list of lifelong ambitions.

I've now updated both the NUnit GUI and MSI Command Launcher applications. The former has an extra try-catch around an event handler for showing details of errors, because of some weird boundary condition. The latter now changes directory to where the executable specified to be run exists, allowing it to use relative paths, etc. if it's a batch file.

Download URLs are the same as ever:

http://www.altervisitor.com/software/NUnit_GUI_Update_Release001.zip

http://www.altervisitor.com/software/MSICommandLauncher.msi

The simplest thing that can possibly (get me out of) work

jsgreenwood — Mon, 06 Sep 2004 22:03:00 GMT

Having spent a few posts on .NET specific tools, I'll get back to more process and architecture oriented topics for a bit now...

As a believer in Agile and XP for delivering software (where appropriate), I subscribe to the "simplest thing that can possibly work" ethos - not overengineering a solution based upon the assumption that as yet unknown requirements will change the defined implementation. An application of YAGNI, if you will. Whilst numerous people in the office have taken this mantra on, I've spotted a lurking anti-pattern that's reared its head a couple of times:

"The simplest thing that could possibly get me out of work"

One of the great things about Agile/XP is that it delivers some control back to the developer, removing the need for business analysts that are basically translator-patterns from business -> technology. This is also one of the approach's problems (which I will discuss in another Blog entry soon); If developers cling to a statement such as "the simplest thing that could possibly work", it means that they are using none of their intelligence, experience, and insight to make decisions. Like most maxims, "The simplest thing..." is a principle to use that cuts to the heart of the majority of cases. It is also a rod for your own back if applied blindly rather than judging the implications of all solutions. In my opinion, a developer is a "professional" who shouldn't mechanically apply rules to derive solutions - a modicum of intelligence is required at each juncture, not a parrot-like ability to repeat a phrase.

I've seen this anti-pattern manifest itself most recently where 2 solid man days were wasted on a task that would've taken 10 minutes if there'd been an understanding of what "simple" implied, and the more elegant solution hadn't been discounted for not involving the simplest steps.

This leads, for me, to two rules:

Whilst in the vast majority of cases, the simplest approach is the best one, there are numerous other cases where something that is only marginally more complicated will clearly give many times the potential going forwards.
The "simplest thing" and the "quickest thing" are quite often very different. "Simple" could mean going through 100 files by hand, manually searching for similar text and overtyping it where needed. "Quick" could mean hitting Ctrl-R in a decent editor and typing in a simple regular expression. I know which of the two options I'd choose. And, unfortunately, I know people that would choose the other option, protecting themselves with the "shield of simple". The "sum of simple" should actually be: simplicity-of-each-step x number-of-steps.

The problem with simplicity is that it exhibits no intelligence.

New Setup & Deploy tool "MSI Command Launcher" released

jsgreenwood — Sat, 04 Sep 2004 00:47:00 GMT

I finished the initial release of my latest (and possibly last for a while) MSI plugin tonight. This one, MSI Command Launcher, extends the ability of custom actions to run user-defined code by:

Supporting the execution of batch files (standard Setup & Deploy packages don't allow this)
Allowing for roll-back of execution based upon the exit code returned from running a script/executable
Provides a standardised Windows GUI for showing command-line input/output, rather than "shelling out" to a DOS box

MSI Command Launcher runs as a proper ProjectInstaller and can be run (i.e. tested) standalone from the command line. Documentation on usage is included in the archive that can be downloaded from: http://www.altervisitor.com/software/MSICommandLauncher.msi

When running, the application appears as follows:

This is a first release of this application, and I've not tested it as much as the others, so feedback would be great. The only problem I know about is if you're writing huge amounts of text to the console in an interactive script (one that requires user input) - the RichTextBox it writes to is pretty slow to update, meaning that input may be expected (and output blocked) before all of the preceeding text has been written to the screen. In applications that are more judicious in their usage of Console.WriteLine, and applications that don't read from the console, this problem won't exist.

If you do use this in your projects, please drop me a mail to let me know - I'd like to track the uptake of all the tools I put out on the Net, so I know which ones to support/extend.

NUnit MSI - an NUnit plugin for Setup & Deploy packages

jsgreenwood — Wed, 18 Aug 2004 01:26:00 GMT

One of the things that's irked me for quite some time is how, even in a mature TDD environment, the tests that are written never seem to make it past the integration server. Yet vast amounts of time gets spent tracing and fixing the differences between environments that cause glitches in execution. The time it takes to spot these errors and the fact that the installation of bad software has already occured makes it a bit like shutting the stable door after the horse has bolted. To me, it became apparent quite some time ago that tests shouldn't just be a means of validating refactorings and amends locally, they should also be a first line of defence when it comes to deploying to and troubleshooting environments (security factors aside).

So, after a few late nights over the last week, the project I've been working on has been finished - NUnit MSI. This application does pretty much what it says on the tin - it sits in MSI installers as a custom action and runs the tests as part of the installation package, deciding whether or not to continue installation based upon success (and the user's input if so desired). It can be downloaded from: http://www.altervisitor.com/software/NUnitMSI.msi, and here's a screenshot to give you an idea:

Screenshot of NUnit MSI

As well as running from within installers, I personally now use it as a custom build action to give me a "green light" that I've not broken anything, rather than having the heavier-weight NUnit GUI running all the time (although it is still useful for persistent errors and larger projects).

The code for NUnit MSI is based directly on NUnit v2.2 (2.2.0), and makes use of all the behind the scenes logic from that, just re-implementing the UI portion of NUnit. Full instructions are included in the archive, and are summarised (in brief) below. As a parting comment, if you use this, please let me know give me feedback - I'm all for extending it, polishing it, making it work as a VS.NET plugin, etc.

Running the NUnitMSI.exe application without any command line
parameters will look in the folder it resides in for the first file with a
.nunit file extension. This will automatically be loaded and the tests run.
If all the tests succeed, the application will pause for one second then
exit successfully. This is the exact same behaviour if it is added to a
Setup and Deploy project.

More advanced command-line usage takes the form:
NUnitMSI.exe [config.nunit] [/queryerrors]

Giving a relative filepath for "config.nunt" will cause the specified NUnit
project file to be loaded, instead of one being probed for.
The "/queryerrors" parameter will cause a dialog to be shown should errors
be detected, asking the user if they wish to continue with installation
regardless. The default is to NOT show this dialog, and for a failed test
to result in the installation rolling back. However, whenever tests fail,
whether this dialog is shown or not, the GUI does NOT automatically exit -
it remains open to allow the user to inspect the errors in more detail.

Usage example: "NUnitMSI.exe ../tests.nunit /queryerrors"

This basic-usage mode allows for NUnit MSI to be integrated with a local
build process as a "post build action", allowing for a succinct graphical
display of the test results, rather than requiring a heavyweight interface
such as the NUnit GUI, or having to parse the output of the command line
installer.

NOTE: once the tests have completed, pressing "F10" on the form will cause
them to run again. This is not documented in the GUI, but is useful when
running the application as an alternative to the standard NUnit GUI
interface;.

NOTE: If the application does not have a ".nunit" file specified, and it
cannot locate one in the execution directory, it will exit automatically
with a status code of "-3". Other status codes are:
    0: OK
    -1: Tests failed
    -2: Invalid ".nunit" file specified