How do we write test automation for ASP.NET?
Part of the responsibility of our team is to write automated regression tests for every ASP.NET feature. It is important to have a comprehensive test bed, since it is used for many years after a product version is released to verify any QFE or subsequent releases does not introduce regressions.
Our team uses an internal automation framework for all testing, and this week we released samples of how to use the "Lightweight Test Automation Framework for ASP.NET". You can find this release in the ASP.NET CodePlex page here. We are very excited to share our tools with the community. If you give it a try, tell us what you think.
What is included in this release?
This release contains samples that show how the
ASP.NET QA team creates automated tests by using the "Lightweight Test Automation Framework for ASP.NET". The release is unsupported, and is provided in the form of samples only.
What is the Lightweight Test Automation Framework for ASP.NET?
This framework was developed and is currently used by the
ASP.NET QA Team to automate regression tests for the product. It is designed to run within an ASP.NET application.
Tests can be written in any .NET Framework language. They use an API to manipulate and verify the DOM of the browser. It supports running tests in Microsoft Internet Explorer, Mozilla Firefox, Apple Safari, and Opera.
Note that even though it can be used to automate testing for any general-purpose Web application, its design was driven by the needs of the QA Team to automate ASP.NET regression tests.
How do I use the samples?
1. Download the Lightweight Test Automation Framework for ASP.NET Samples from the Web site and then open it in Visual Studio 2008. The Web site contains a small application and tests to automate it. The following illustration shows the Web site structure.
2. Navigate to Home.aspx to interact with the Web site. To log in, use the user name ValidUser and use any string for the password. The page contains a GridView control that lets the user edit, delete, and insert data about school classes and grades. The following illustration shows a screen shot of the sample application.
3. Navigate to \test\Default.aspx. This loads the testing UI, shown in the following illustration:
4. Select tests from the tree view and then click Run Tests. The tests will run and display their progress in the right-hand frame.
5. To examine the test code, browse to the App_Code\Tests folder, and then open any file to inspect the test source code.
How do I debug test errors?
When a test fails, it is marked in red in the test tree. To examine the log, click the test name. The log information is displayed as shown in the following illustration:
The log contains the stack trace of the test and the DOM that was captured by the framework at the moment of failure.
You can also have the test framework write a simple log to disk. To do so, select the Write Log to Disk check box before you run the tests. Note that this feature requires write access to the Web site’s root folder.
How do I use it with my own Web site?
1. Copy Microsoft.Web.Testing.Lightweight.dll into the Bin folder of your Web site. If you are testing a Web application, add the DLL as a reference to the project.
2. Copy Default.aspx and DriverPage.aspx into a folder in the Web site. By convention, the ASP.NET QA team uses the Test folder, but you can use any folder that you want. These are the pages that are required in order to run the tests.
3. If you are testing a Web site project, create tests in the App_Code folder. If you are testing a Web application project, you can create tests in any folder.
How do I use query-string arguments?
The framework accepts the following query-string arguments that are useful for filtering and running tests:
- Tag=<tag_identifier>
- Selects all tests that are marked with the WebTestTagAttribute and that have the specified identifier. For example, to automatically select a method that is marked with the [WebTestTag("DataScenarios")] attribute, you can navigate to the following URL:
http://server/site/test/Default.aspx?tag=DataScenarios
By default, the names of all classes that are marked with the WebTestClassAttribute and the name of all methods that are marked with the WebTestMethodAttribute are already identifiers.
- You can specify multiple tag identifiers by concatenating them with the “@” symbol, as shown in the following example:
http://server/site/test/Default.aspx?tag=Pri0@MyTestMethod
This example shows how to automatically select all tests that have the [WebTestTag("Pri0")] attribute and additionally to select the test method named “MyTestMethod”.
- Filter=[true|false]
- Passing true causes all unselected tests to be removed from the test tree.
- Run=[true|false]
- Passing true will start running the selected tests on load.
Additional Resources
For more information, see the following resources:
- The ASP.NET QA Team blog, where you can find additional samples.
- The ASP.NET QA Road Map document, which can be found within the ASP.NET CodePlex page. This document describes upcoming features.
- The ASP.NET CodePlex page, where you can post questions and feedback.
- Federico Silva Armas
ASP.NET QA Team