Automated UI Testing with Project White

Try it, or else...A co-worker turned me onto Project White, an automated UI testing framework by ThoughtWorks. This along the same lines as NUnitForms and other automated systems. It's basically Selenium for WinForms (which rocks in its own right) so I thought I would dig more into White (it has support for WPF as well, but I haven't tried that out yet). It was good timing as we've been talking and coming up with strategies for testing and UI testing is a big problem (and it is everywhere else based on people I've talked to).

The White library is nice and simple. All you really need to do is add in the Core.dll from White and your unit test framework and write some tests. I tested it with MbUnit but any framework seems to work. Ben Hall posted a blog entry about White along with some sample code. This, combined with the library got me started.

As Ben did, I created a simple application with a single form and started to write some tests. I couldn't use Ben's complete sample as it was written for VS2008 and I only had VS2005 for my testing. No problem. You can use White with VS2005, but you'll need the 3.0 framework installed.

I came across the intial problem with testing though. The first test that failed left the window up on the screen. This was an issue. I also wrote the same test Ben did, looking for a non-existant form, which appropriately threw a UIActionException. The test passed as it threw the exception I was looking for, but again the form was up on the screen. The Application.Kill() method wasn't being called if the test would fail or an exception was thrown. Ben's method was to put a call to Application.Kill in the [TearDown] method on the test fixture. This is great but I'm not a big fan of [SetUp] and [TearDown] methods. Another option was to surround each test with a try/catch/finally and in the finally code call out to the Application.Kill() method. This was ugly as I would have to do this on every test.

Following Ben's example I created a WhiteWrapper class which would handle the White library features for me. I made it implement IDisposable so I could do something like this:

  1 using(WhiteWrapper wrapper = new WhiteWrapper(_path))
  2 {
  3 	...
  4 }
  5

I also added a method to fetch me a control from the main window (using a Generic) so I could grab a control, execute a method on it (like .Click()) and check the result of another control (like a Label). Note that these are not WinForm controls but rather a White wrapper around them called UIItem. This provides general features for any control like a .Click method or a .Text property or items in a listbox.

Here's my WhiteWrapper code:

  1 class WhiteWrapper : IDisposable
  2 {
  3 	private readonly Application _host = null;
  4 	private readonly Window _mainWindow = null;
  5 
  6 	public WhiteWrapper(string path)
  7 	{
  8 		_host = Application.Launch(path);
  9 	}
 10 
 11 	public WhiteWrapper(string path, string mainWindowTitle) : this(path)
 12 	{
 13 		_mainWindow = GetWindow(mainWindowTitle);
 14 	}
 15 
 16 	public void Dispose()
 17 	{
 18 		if(_host != null)
 19 			_host.Kill();
 20 	}
 21 
 22 	public Window GetWindow(string title)
 23 	{
 24 		return _host.GetWindow(title, InitializeOption.NoCache);
 25 	}
 26 
 27 	public TControl GetControl<TControl>(string controlName) where TControl : UIItem
 28 	{
 29 		return _mainWindow.Get<TControl>(controlName);
 30 	}
 31 }
 32

And here are the refactored tests to use the wrapper (implemented via a using statement which makes using the library fairly clean in my test code):

  1 public class Form1Test
  2 {
  3 	private readonly string _path = Path.Combine(Directory.GetCurrentDirectory(), "WhiteLibSpike.WinForm.exe");
  4 
  5 	[Test]
  6 	public void ShouldDisplayMainForm()
  7 	{
  8 		using(WhiteWrapper wrapper = new WhiteWrapper(_path))
  9 		{
 10 			Window win = wrapper.GetWindow("Form1");
 11 			Assert.IsNotNull(win);
 12 			Assert.IsTrue(win.DisplayState == DisplayState.Restored);
 13 		}
 14 	}
 15 
 16 	[Test]
 17 	public void ShouldDisplayCorrectTitleForMainForm()
 18 	{
 19 		using (WhiteWrapper wrapper = new WhiteWrapper(_path))
 20 		{
 21 			Window win = wrapper.GetWindow("Form1");
 22 			Assert.AreEqual("Form1", win.Title);
 23 		}
 24 	}
 25 
 26 	[Test]
 27 	[ExpectedException(typeof(UIActionException))]
 28 	public void ShouldThrowExceptionIfInvalidFormCalled()
 29 	{
 30 		using (WhiteWrapper wrapper = new WhiteWrapper(_path))
 31 		{
 32 			wrapper.GetWindow("Form99");
 33 		}
 34 	}
 35 
 36 	[Test]
 37 	public void ShouldUpdateLabelWhenButtonIsClicked()
 38 	{
 39 		using (WhiteWrapper wrapper = new WhiteWrapper(_path, "Form1"))
 40 		{
 41 			Label label = wrapper.GetControl<Label>("label1");
 42 			Button button = wrapper.GetControl<Button>("button1");
 43 			button.Click();
 44 			Assert.AreEqual("Hello World", label.Text);
 45 		}
 46 	}
 47 	
 48 	[Test]
 49 	public void ShouldContainListOfItemsInDropDownOnLoadOfForm()
 50 	{
 51 		using (WhiteWrapper wrapper = new WhiteWrapper(_path, "Form1"))
 52 		{
 53 			ListBox listbox = wrapper.GetControl<ListBox>("listBox1");
 54 			Assert.AreEqual(3, listbox.Items.Count);
 55 			Assert.AreEqual("Red", listbox.Items[0].Text);
 56 			Assert.AreEqual("Versus", listbox.Items[1].Text);
 57 			Assert.AreEqual("Blue", listbox.Items[2].Text);
 58 		}
 59 	}
 60 }

The advantage I found here was handling exceptions and unknown states. For example in the last test, ShouldUpdateLabelWhenButtonIsClicked I ran the test before I even had the controls on the form. The test failed but it didn't hang or crash the system. That's what the IDisposable gave me, a nice way to always clean up without having to remember to create a [TearDown] method.

One of the philosophical questions we have to ask here is when is this kind of testing appropriate? For example, if I have good presenters I can test these kind of things with mocked out views and presenter/model tests. So am I duplicating effort here by testing the UI directly? Should I get my QA people to write these kind of tests? There's a long discussion to have in your organization around this so it's not just a "tool problem". You need to dig deep into what you're testing and how. At some point, you begin to divorce yourself from behaviour driven development and you end up testing UI edge cases and integration from a UI perspective. If your UI doesn't line up with your domain, how do you reconcile this? There are probably more questions than answers for this type of thing and software design is more art than science. The answer "it depends" goes a long way, but don't try to solve your business or design problems with a tool. There is no silver bullet here, just a few goodies to help you along the way. It's you who needs to decide what's appropriate for the situation and how much time, money, and resources you're going to invest in something.

The library works pretty good and I'm happy with the test code so far. We'll have to see now how it deals with far more complex UIs (we have things like crazy 40-column grids with all kinds of functionality). Back later on how that goes. In the meantime, check out Project White here on CodePlex to help you with your automated UI testing.

1 Comment

Comments have been disabled for this content.