Unit Testing with Silverlight

One of the important capabilities we shipped with the Beta1 release of Silverlight 2 was a unit test harness that enables you to perform both API-level and UI-level unit testing.  This testing harness is cross browser and cross platform, and can be used to quickly run and verify automated unit tests:

In addition to shipping this unit test harness for Silverlight, we also shipped the source to ~2,000 unit tests built with it that provide automated coverage for the Silverlight control source that we also shipped under a permissive license (you can take the control source, modify it, run the unit tests to verify the behavior, then re-ship the controls however you want).

Learning How to Unit Test Silverlight

Jeff Wilcox (who developed the Silverlight unit test framework and harness) has a great blog post that talks about how to add a Silverlight Unit Test project to a solution here. You can download the chat application that he shows testing from this expression blend blog post tutorial I did last month.  You can also watch this cool video post that Jeff created where he walks through the unit test framework and test cases we've shipped.

As Jeff shows in his post, you can now add a "Silverlight Test Project" to your Visual Studio solution which encapsulates unit tests for an application you are working on:

You can then add unit test classes to the test project that test APIs or simulate UI action within the Silverlight controls (simulate button clicks, etc).

You can then run the test project and execute the tests within it to verify and report their status.

Jeff's test framework automatically provides a browser based test harness and reporting system (which means you can run it on any browser/OS combination that Silverlight runs on):

Jeff's test framework supports quickly re-setting controls after each test (and avoids needing to re-launch a new browser instance for each test cases - which makes it really fast).

You can quickly rip through hundreds or thousands of automated tests in seconds:

Green results mean the tests passed.  Red results flag that a test case failed and log the assertion failure and/or runtime exceptions that occurred.

Summary

If you've ever struggled to try and come up with a strategy for doing automated unit testing or TDD with AJAX applications, I think you'll find Silverlight provides some much nicer test options.  Using Visual Studio you can also separate your tests into a separate project in your solution, and you do not need to embed the tests within your Silverlight application in order for them to run.

In addition to supporting the above unit test harness and framework, we are also going to support UI automation APIs with the final release of Silverlight 2.  These will enable accessibility scenarios (allowing screen readers to work with Silverlight and enable Section 508 compliance of Silverlight applications).  These UI automation APIs will also enable UI testing scenarios where you can build end to end browser UI automation that simulates real mouse and keyboard interactions and enables automated end to end experience testing.  The combination should enable you to build much more solid and maintainable RIA solutions.

Hope this helps,

Scott

P.S. For more tutorial posts and links on Silverlight 2, check out my new "Silverlight Tips, Tricks, Tutorials and Links" page.

Published Wednesday, April 02, 2008 2:01 AM by ScottGu

Comments

# Unit Testing with Silverlight - ScottGu's Blog

Wednesday, April 02, 2008 5:39 AM by Unit Testing with Silverlight - ScottGu's Blog

Pingback from  Unit Testing with Silverlight - ScottGu's Blog

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 5:43 AM by Gill Cleeren

Really valuable post, thanks.

# Unit Testing with Silverlight

Wednesday, April 02, 2008 6:35 AM by Unit Testing with Silverlight

Trackback from SilverlightShow.net

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 6:40 AM by Granville Barnett

Nice post :-)

Btw, you really should stop talking to yourself on the chat client :-)

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 6:55 AM by Chris Hardy

Having something that dumps unit testing results to a html page for WebForms/MVC would be great - Rspec does it with Rails and it looks great and I got rather jealous.

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 7:24 AM by Christian Pfeil

Thanks for the tutorial!

Christian Pfeil

# ScottGu: Unit Testing with Silverlight

Wednesday, April 02, 2008 7:26 AM by DotNetKicks.com

You've been kicked (a good thing) - Trackback from DotNetKicks.com

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 8:01 AM by Håkan Forss

Nice!

Will we see something like this for standard WPF applikations as well? UI automation APIs for WPF apps and WinForm apps would be very usefull for testing.

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 8:05 AM by daveloper

Would this mean that the unit tests are also send down to the client, since the testing harness runs locally?

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 8:29 AM by chhaysambo

Good post, thanks.

# AJAX coding school » Blog Archive » AJAX Tutorials [2008-04-02 12:43:55]

Pingback from  AJAX coding school  » Blog Archive   » AJAX Tutorials [2008-04-02 12:43:55]

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 9:01 AM by Alvin Ashcraft

Thanks Scott. This is some great info.

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 9:32 AM by nmosafi

Hey Scott

This is great for testing View based components and controls.  However ideally I would love to be able to test my Model/Controller/Utility Classes within Visual Studio / MSTest like I can with code not built on the Core CLR.

TestDriven.NET have a way of doing this, although AFAIK it doesn't work with the latest Silverlight 2 Beta.

It would be great to be able to host CoreCLR assemblies within a CLR process and have the two communicate (a bit like the SL's HTML Bridge).  Do MS have any plans to provide this?

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 10:20 AM by Gill Cleeren

Great info, thanks.

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 10:43 AM by Aaron Jensen

How can I do this from my msbuild file? On a continuous integration server? Does it *need* a browser?

# Brian Noyes / WPF Update on .NET Rocks « Mark Gilbert’s Blog

Pingback from  Brian Noyes / WPF Update on .NET Rocks « Mark Gilbert’s Blog

# Daily Find #50 | TechToolBlog

Wednesday, April 02, 2008 12:31 PM by Daily Find #50 | TechToolBlog

Pingback from  Daily Find #50 | TechToolBlog

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 12:52 PM by BringerOD

Good to see more example of unit testing.

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 1:00 PM by jwilcox

'daveloper',

To your question about whether unit tests are also sent down to the client, they are only if you include the tests in the same Silverlight application project.

I'd recommend creating a separate test project (details in the tutorial), your tests are then in one .Xap file - Test.Xap - and your app is in another, App.Xap.

-Jeff

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 1:28 PM by jwilcox

Håkan Forss,

The UI testing that's being done with this current framework is actually testing the wired up APIs- verifying that events sink, elements are in the visual tree, etc.  At this time there isn't an actual mouse/keyboard automation component there.  

That said, I think many people are interested in what it would take to perform UI automation for WPF and WinForms apps, I'll share your feeback with some folks.  Thanks!

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 1:36 PM by jwilcox

nmosafi,

That's good feedback about being able to run Core CLR tests from within Visual Studio.

One question I have for you: if you had a way to test "Silverlight" applications right in Visual Studio, you'd only be able to test basic APIs and class libraries: the HTML DOM bridge, the entire rich presentation framework in Silverlight, all those things would -not- be accessible to those tests.  Would that be acceptable?  

In my opinion, I'm more interested in exercising those Silverlight-specific features in the browser, since I can just as easily compile the same source files in my regular desktop CLR and run them as regular Visual Studio test projects to verify that my code is working fine.

- Jeff

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 2:36 PM by Jacek Ciereszko

Thanks for the tutorial!!!

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 3:59 PM by daveloper

jwilcox,

Thanks for clearing that up!

TDD with ASP.NET UI components isn't as intuitive compared to logic layers, this really helps out.

# Silverlight Cream for April 2, 2008 -- #242

Wednesday, April 02, 2008 6:53 PM by Community Blogs

Michael Sync posted a list of SL2B1 problems, Rob Houweling produced a Motion Blur demo, LeeOnTech with

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 7:34 PM by Tom

How do I get the cool background of the editor that you are showing above in an example. Off the topic..sorry :-)

# Silverlight 2 has built in Unit Testing « An Original Idea

Pingback from  Silverlight 2 has built in Unit Testing « An Original Idea

# Unit Testing in Silverlight

Wednesday, April 02, 2008 11:12 PM by Frank La Vigne

# re: Unit Testing with Silverlight

Wednesday, April 02, 2008 11:59 PM by Annoying Person

Why are you using a Mac? Aren't Microsoft people supposed to hate them? Are you informally admitting Microsoft's inferiority? Or did you just want to demonstrate the cross-platform capabilities of Silverlight?

# AlanLe.net » links for 2008-04-03

Thursday, April 03, 2008 4:36 AM by AlanLe.net » links for 2008-04-03

Pingback from  AlanLe.net » links for 2008-04-03

# re: Unit Testing with Silverlight

Thursday, April 03, 2008 9:40 AM by Ashutosh Bhardwaj

Hi Scott,

Is there any tutorial on the UNIT TEST functionality built within VS 2008? Are you planning to take up any tutorials on the same?

# re: Unit Testing with Silverlight

Thursday, April 03, 2008 2:33 PM by dave.parslow

Hi Scott,

Great job on providing the unit test for Silverlight! I think this will really help the .net community adopt unit testing and TDD as common practices by seeing what Micrsoft themselves do.  When can we expect the unit tests for ASP.NET MVC?  Last I heard you needed to work on the mock framework that ASP.NET MVC was using.

Dave Parslow

# re: Unit Testing with Silverlight

Thursday, April 03, 2008 2:57 PM by Duckie

Helped a lot! Thanks. Back to work.

- There should be a Scott fan-site.

# Ni Silverlight se libra de las pruebas unitarias!!

Thursday, April 03, 2008 4:41 PM by Yo sólo pasaba por aquí pero ya que estoy....

Para los amantes de Silverlight ya tenéis cómo podéis hacer vuestras pruebas unitarias con este tipo

# re: Unit Testing with Silverlight

Friday, April 04, 2008 5:05 AM by ScottGu

Hi Ashutosh,

>>>>>> Is there any tutorial on the UNIT TEST functionality built within VS 2008? Are you planning to take up any tutorials on the same?

Stephen Walther has a great post here that should help: weblogs.asp.net/.../tdd-test-driven-development-with-visual-studio-2008-unit-tests.aspx

Thanks,

Scott

# re: Unit Testing with Silverlight

Friday, April 04, 2008 5:05 AM by ScottGu

Hi Dave,

>>>>> Great job on providing the unit test for Silverlight! I think this will really help the .net community adopt unit testing and TDD as common practices by seeing what Micrsoft themselves do.  When can we expect the unit tests for ASP.NET MVC?  Last I heard you needed to work on the mock framework that ASP.NET MVC was using.

There will be an update to ASP.NET MVC later this month that makes testing controllers much easier, and won't require mocking any objects to-do so.

Thanks,

Scott

# re: Unit Testing with Silverlight

Friday, April 04, 2008 5:06 AM by ScottGu

To Everyone,

Make sure to check out Jeff's answers above - he has jumped in and answered several of the earlier questions on unit testing.

Thanks,

Scott

# re: Unit Testing with Silverlight

Friday, April 04, 2008 7:31 AM by vik20000in

I am really linking Silverlight

# re: Unit Testing with Silverlight

Friday, April 04, 2008 9:25 AM by nmosafi

Hi Jeff

I think a combination of both types of testing would be good.  If tests can be run from MSTest then I can integrate them into my build process.  Those tests which have a dependency on the browser to run would not be able to run via MSTest and would require the browser.

Also a good point that you could compile the same source files as a normal CLR assembly and test them that way.  I might try that!

Neil

# re: Unit Testing with Silverlight

Friday, April 04, 2008 12:45 PM by ScottGu

Hi Tom,

>>>>>>> How do I get the cool background of the editor that you are showing above in an example. Off the topic..sorry :-)

Here is a pointer to my VS settings file if you want the same VS color scheme: www.scottgu.com/.../vssettings.zip

Hope this helps,

Scott

# re: Unit Testing with Silverlight

Sunday, April 06, 2008 9:18 AM by Juan María

I all:

YOu can read this post in spanish here:

thinkingindotnet.wordpress.com/.../tests-unitarios-con-silverlight

# Tests unitarios con Silverlight « Thinking in .NET

Sunday, April 06, 2008 9:18 AM by Tests unitarios con Silverlight « Thinking in .NET

Pingback from  Tests unitarios con Silverlight « Thinking in .NET

# Silverlight Reading « Tales from a Trading Desk

Sunday, April 06, 2008 4:40 PM by Silverlight Reading « Tales from a Trading Desk

Pingback from  Silverlight Reading « Tales from a Trading Desk

# VSTS Links - 04/07/2008

Monday, April 07, 2008 9:54 AM by Team System News

Steve Andrews on Tech-Ed with the Blogfather. Tim Huffam on VSTS not recognizing unit tests - Run Tests...

# Wöchentliche Rundablage: ASP.NET MVC, Silverlight 2, .NET, RegEx, .NET, Icons, CSS, UI | Code-Inside Blog

Pingback from  Wöchentliche Rundablage: ASP.NET MVC, Silverlight 2, .NET, RegEx, .NET, Icons, CSS, UI | Code-Inside Blog

# re: Unit Testing with Silverlight

Monday, April 07, 2008 3:18 PM by Troy Gould

Why only be able to run and see results of unit tests from the browser?  It would be useful to be able run from the command line and generate a results file so that I can include in my CI process!  Please don't assume we are all vendor-locked into Team System for CI either.

# Resumo da semana - 07/04/2008

Monday, April 07, 2008 9:12 PM by Console.Write(this.Opinion)

Resumo da semana - 07/04/2008

# Finds of the Week - April 6, 2008 » Chinh Do

Monday, April 07, 2008 11:27 PM by Finds of the Week - April 6, 2008 » Chinh Do

Pingback from  Finds of the Week - April 6, 2008 » Chinh Do

# re: Unit Testing with Silverlight

Tuesday, April 08, 2008 1:28 AM by Chris Chandler (developingchris)

Nice post, ready to give it the old tire kick and test drive.

# re: Unit Testing with Silverlight

Friday, April 11, 2008 2:34 PM by Raviraj

Hi,

Great post...

I have one question though...is there a place where i can see sample code for testing something asynchronously...something like testing the play() for a MediaElement...

# re: Unit Testing with Silverlight

Monday, April 14, 2008 5:55 PM by Jeff Wilcox

Raviraj,

I'll post an asynchronous tutorial to my blog soon.  In the meantime, try:

1. Having your test class inherit from SilverlightTest base, inside Microsoft.Silverlight.Testing namespace

2. Marking the test method for asynchronous testing with the [Asynchronous] attribute, in the same namespace

3. Inside your test call the play() method on your MediaElement.  Hook up to an appropriate event for the player, OR use a DispatcherTimer, etc., to verify that everything is OK and make any assertions.  In the method with the verification at the end of testing, call TestComplete().  This is a method that is exposed by the SilverlightTest base class.

Hope this helps.

-Jeff

# re: Unit Testing with Silverlight

Tuesday, April 15, 2008 9:47 AM by Edward Miller

Do you have any estimate on when the UI automation interface will be available?  We need this ASAP to implement browser based tests of Silverlight apps which we didn't build.

# re: Unit Testing with Silverlight

Wednesday, April 16, 2008 6:56 PM by Issam Qadan

Very useful post, along with the comments. On the subject of testing Silverlight, are there any UI Test Harness tool from Microsoft that is publicly available for driving UI test Automation?

# re: Unit Testing with Silverlight

Wednesday, April 30, 2008 10:52 AM by resveratrol

I have to say, coming from adobe flash ui development, this is quite a leap regarding testing.

I am very happy with it so far.

A

Leave a Comment

(required) 
(required) 
(optional)
(required)