Due to some test activities that I have been engaged on lately I have had to educate myself in regards to web standards. One of our PM's sent me this link that I want to share out with any testers out there that are not aware of it. This is a document found within the Massachusetts government website, and I am not sure if it is somehow special for that state (it's from 2005, so a little old). Even though it is not a replacement for Section 508, I liked the way it is presented because for each standard it explains "why" the standard is needed and "what' it means with examples. I include the list of standards that are defined:

1. Layout and Design

• Standard 1.1: A state agency web page must use HTML markup tags according to industry standards.
• Standard 1.2: A state agency web page should control presentation with style sheets, but the web pages must be organized so they are readable without requiring an associated style sheet.
• Standard 1.3: A state agency web page should avoid using frames.
• Standard 1.4: State agencies must ensure that their web pages are usable when scripts, applets, or other programmed objects are turned off or are not supported.
• Standard 1.5: Forms designed to be completed online, and other interactive interface elements, must be accessible by people using assistive technology.
• Standard 1.6: When a timed response is required, the user must be provided the opportunity to request additional time to continue working before the time-out occurs.

2. Navigation

• Standard 2.1: A state agency web page must use clear and consistent navigation mechanisms.
• Standard 2.2: A state agency must ensure that users are able to interact with web page elements in a device independent manner.

3. Graphics and Sound

• Standard 3.1: A state agencyweb page must provide a text equivalent for every non-text element.
• Standard 3.2: A state agency web page must provide synchronized auditory and readable text descriptions of the important information of the visual track of a multimedia presentation.
• Standard 3.3: A state agency web page must provide a text equivalent for information provided in audio format.
• Standard 3.4: A state agency web page that uses motion must ensure that the motion is integral to the content of the site, user-controlled, and limited to three cycles and then stopped automatically.
• Standard 3.5: A state agency must ensure that the use and selection of color do not affect the information conveyed on a page.
• Standard 3.6: Client-side image maps are recommended. If server-side image maps must be used, provide redundant text links for each active region.

4. Content Requiring Additional Software

• Standard 4.1: All information published on a state agency web page must be published in HTML, whenever possible, to eliminate the need for additional software.
• Standard 4.2: A state agency that offers files to download in compressed format (Zip files, for example) must also provide the same information in its uncompressed format or as a self extracting file.

5. File Size

• Standard 5.1: A state agency must optimize files to improve download time.

6. Web Accessibility Statement

• Standard 6.1: A state agency web site must provide a link to a web accessibility statement.

7. Web Site Validation

• Standard 7.1: A state agency must validate web content against these Standards prior to posting and at regular intervals after posting.

Federico Silva Armas
SDET, ASP.NET QA Team

Download : Episode 4

In the fourth installment of the ASP.NET QA Podcast Federico and Matthew discuss “The Good,” “The Bad,” and “The Ugly” of the history of the ASP.NET QA Team. Discussion ranges from the dark ages, to the renaissance, to the industrial revolution, to the “21st and half century”.  Join them as they discuss the lessons learned and how the team has improved over time.

• The team released the April update to the Lightweight Test Automation Framework.
• “The Dark Ages” ASP.NET 1.0/2.0
  • During this time the team was very focused on automating everything.
  • (disadvantage) This approach is not good with a changing Spec.
  • (disadvantage) Bugs were being found to late in the process.
  • (advantage) Excellent automation coverage.
• “The Renaissance” Atlas/Microsoft Ajax
  • The division moved to a feature crew model for development and QA.
  • (advantage) The team adopted a heavy unit testing practice. Increase in code coverage.
  • (disadvantage) The QA team still worked has it had before. It would automate everything is less time.
• “The Industrial Revolution” ASP.NET 3.5
  • The QA team began to adopt the Feature crew model and started the process of agile testing with exploratory testing.
  • (advantage) Bugs were being found at the beginning of the process.
  • (disadvantage) The team still wrote a large amount of automation.
• “21st and half century” ASP.NET 3.5 SP1/ASP.NET 4.0/vNext
  • The team has really started moving away from automating everything and is spending more time investigating user scenarios.
  • The team has formed its own unique blend of agile, scrum, and any other buzz word project management theory.
  • (advantage) Creating and automating sample applications provides feedback for real world customer scenarios.
  • (disadvantage) The team is young and is still learning the best practices.
  • (Struggle) Costing has become less of a clear cut process.
  • (Struggle) The is still a barrier between the QA and Dev teams.
  • (Struggle) No one wants the blame when a bug is found.

Links from the show :

• Lightweight Test Automation Framework April Release
• Federico’s Email
• Matthew’ Email
• ASP.NET QA Team Blog

The April release of the Lightweight Test Automation Framework for ASP.NET has just been posted and you can download it here. For this release, the team has worked hard to include the following bug fixes and new features. Please continue to give us your feedback as many of the fixes and features are based what we have heard from the community.

Improvements to the user interface

• A new look has been given to the test name when it passes or fails.  There is both color and visual queues that indicate weather a test has passed or failed.  Failed test names also appear slightly larger to help them stand out.
• There is now a “Run Failed Tests” button. This button will open a new browser window that will select and run only the failed tests, for easy verification of fixes.

The ability to automate popup windows

In previous versions of the framework there was no way to verify the contents of a popup window.  With this release we have far better support for opening and verifying the contents of popup windows.

HtmlPage page = new HtmlPage("MyPage.aspx");
page.Elements.Find("OpenPopup").Click();
// get popup window
HtmlPage popup = page.GetPopupWindow(1);
// verify title of popup
Assert.AreEqual("This is the Popup Page", popup.Elements.Find("h1", 0).GetInnerText());

The GetPopupWindow method returns a HtmlPage object that is representative of the window at that index.  This is pulled from the collection that is maintained by the framework with index zero being the main, or starting, window. In this example index zero is MyPage.aspx, while index one is the popup window. 

The ability to find elements by partial attribute values

In previous versions of the framework when finding elements on a page you could only use the ID attribute to match against.  So if you wanted to match all the elements that had a CSS class applied to them, as is common in jQuery, you had to supply that whole value for the class attribute.  So, in previous versions of the framework if you wanted to find elements that had a CSS class applied to them you had to specify the whole value for the class attribute.  Meaning that if the element has more than only class and you search based on only one class that element would not be returned.  Now you have the ability to match based on any part of the value for the attribute. Below is an example of how to find elements that have the CSS Class “blue” applied to them.

HtmlElementFindParams find = new HtmlElementFindParams();
find.Attributes.Add("class", "blue", MatchMethod.Contains);
ReadOnlyCollection<HtmlElement> elements = page.Elements.FindAll(find);

Assembly name change

• The assembly name has been changed from “Microsoft.Web.Testing.Light.dll” to “Microsoft.Web.Testing.Lightweight.dll” The namespaces have not changed.

Download : Episode 3

In the third installment of the ASP.NET QA Podcast Federico and Matthew interview Mark Berryman, a test lead on the ASP.NET QA Team.  Join them as they learn the ins and outs of test management through Mark’s eyes.

• Release of the Mobile Browser Database by Live Dublin.
• April refresh of the Lightweight Test Automation Framework.
• Interview with Mark Berryman.
  • What is the difference between Test Manager and Test Lead.
  • A little history of Mark and how he came to join the ASP.NET QA team.
  • Responsibilities of a Test Lead / What Mark is working on.
  • What Mark looks for in a tester on his team.
  • Challenges Mark faced when he was an IC (individual Contributor).
  • What Mark would change on items he has tested in the past.
  • How Mark Manages risk on his team.
  • Who is Marks favorite person in the whole world.

Links from the show :

• Mobile Browser Database
• Federico’s Email
• Matthew’ Email
• ASP.NET QA Team Blog

Download : Episode 2

In the second installment of the ASP.NET QA Podcast Federico and Matthew discuss MIX 09 announcements and the ins and outs of supporting various browser versions on the ASP.NET QA team.

• MIX 09 Conference announcements
  • MVC RTM 1.0
  • Silverlight 3.0
  • Web Application Installer
  • Web Application Gallery
• What browsers the team tests on and how they decided which browsers to test.
• What it means when the team says they support a browser.
• How the team manages the risk of not testing on every possible browser.
• The timeline for testing features on multiple browsers.
• The infrastructure the team has to test on multiple browsers.
• How the team keeps up with the fast growing browser market, such as the release of four major browsers in just a few months.

Links from the show :

• ASP.NET MVC RTM 1.0
• Silverlight 3.0
• Web application installer
• Web application gallery
• Federico’s Email
• Matthew’ Email
• ASP.NET QA Team Blog

I was recently tasked to prototype a way to test our ControlDesigners without using VisualStudio. For those of you that are not familiar with ControlDesigners here is an overview. In short, ControlDesigners is what give an Asp.Net runtime control a design time experience when hosted inside VisualStudio. Traditionally, our team (and most teams in DevDiv) prefer to automate all of VisualStudio to get regression coverage of designer features.

Now, ControlDesigners that were written for Asp.Net 1.0 and 2.0 do not have unit tests (yeah, our team didn't see the light until later), so our QA team has had to maintain a big automation bed to verify regressions in our ControlDesigners that is generally a pain to write (constant product and framework changes, coupled with the fact that the Visual Web Developer team rewrote the design surface twice make VisualStudio UI testing very challenging). So one approach that we are investigating is to see if we can add unit tests for our old ControlDesigners.

You can download a simple prototype of how such testing may look like HERE.

This is very early, but it gave us good confidence that we could probably test most of our core controls. Here is an example of how a very simple test looks like:

   1: [TestMethod]

   2: public void VerifyRenderingWithId()

   3: {

   4:     TestDesigner designer = new TestDesigner();

   5:     HyperLink control = new HyperLink();

   6:     control.ID = "HyperLink1";

   7:  

   8:     designer.VerifyRendering(@"<a id=""HyperLink1"" Href=""url"">[HyperLink1]</a>", control);

   9: }

There is also a more advanced test to verify the design time rendering of  a GridView control that is bound to a DataSource control:

   1: [TestMethod]

   2: public void VerifyRenderingWithSchema()

   3: {

   4:     TestDesigner designer = new TestDesigner();

   5:  

   6:     TestDataSourceViewSchema schema = new TestDataSourceViewSchema();

   7:     schema.AddField(new TestDataSourceFieldSchema());   

   8:     schema.AddField(new TestDataSourceFieldSchema());

   9:     designer.AddDataSourceControl("DataSource1", schema);

  10:  

  11:     GridView control = new GridView();

  12:     control.ID = "GridView1";

  13:  

  14:     ControlDesigner controlDesigner = designer.AddDataBoundControl(control, "DataSource1");

  15:  

  16:     string expectedHtml = @"<table cellspacing=""0"" rules=""all"" border=""1"" id=""GridView1"" style=""border-collapse:collapse;"">

  17:         ... 

  18:         cut for space

  19:         ...

  20:         ";

  21:     designer.VerifyRendering(expectedHtml, controlDesigner);

  22: }

Some comments regarding this project:

• The idea is that most ControlDesigners make calls into our runtime controls, which makes the designer vulnerable to changes to our runtime code. Having this level of simple regression tests can help us catch changes that could break the design time experience of controls.

• Notice that I am making simple string comparisons to verify big chunks of HTML code. Internally, we use a TagDiffer tool that helps in abstracting common rendering changes that should not break the test (like attribute ordering). I had to modify the project to not include these binaries since I am not sure that I am allowed to release it.

• I did a little bit of research to see if I could find a mock implementation of the designer that would help me with this. I could not find anything, but if anybody knows of a tool that we can use or has some pointers of something similar I would really appreciate it.

• There are many services that are not implemented, so plugging other controls into this may or may not work :P

Any feedback you have is welcome.

Federico Silva Armas
SDET ASP.NET QA Team

 Update: With the newest release of LTAF there is a simplified way of attaching to events that can be used to generate a custom log. Read more about it in this post.

One question that I got from those using the Lightweight Test Automation Framework is how to create your own log. In this post I’ll go over the changes you need to make to create your own log that outputs an html table with the tests that ran and their results. The output will be really simple, but it is meant as a sample.

1. The first thing is to create a class that inherits from TestcaseExecutor. This class lets you override methods that are called when each test case starts and ends, and we’ll use it to create our custom log. We’ll also override a method that is called at the end of all test run, where we will write the log to disk (notice that in this sample asp.net needs write access to the Test directory)

   1: public class LogTestcaseExecutor: TestcaseExecutor

   2: {

   3:     private readonly string _testPath;

   4:     private StringBuilder _log;

   5:  

   6:     public LogTestcaseExecutor(string testPath)

   7:         : base()

   8:     {

   9:         _testPath = testPath;

  10:         _log = new StringBuilder();

  11:     }

  12:  

  13:     protected override void OnTestcaseExecuted(TestcaseExecutionEventArgs e)

  14:     {

  15:         string tableRow = @"

  16: <tr style='"background-color:{2};"">

  17:     <td>{0}</td>

  18:     <td>{1}</td>

  19: </tr>

  20: ";

  21:         _log.AppendFormat(tableRow,

  22:             e.WebTestName,

  23:             (e.Exception == null) ? "&nbsp" : e.Exception.Message,

  24:             (e.Passed) ? "green" : "red");  

  25:         

  26:         base.OnTestcaseExecuted(e);

  27:     }

  28:  

  29:     protected override void OnTestRunFinished(TestRunFinishedEventArgs e)

  30:     {

  31:         string log = @"

  32: <html>

  33:     <body>

  34:         <table style='"width:100%;border-style: solid;border-width: 1px;"">

  35:             <tr>

  36:                 <td>Testcase</td><td>Error</td>

  37:             </tr>

  38:             {0}

  39:         </table>

  40:     </body>

  41: </html>";

  42:  

  43:         File.WriteAllText(Path.Combine(_testPath, "RunLog.htm"), String.Format(log, _log.ToString()));

  44:         

  45:         base.OnTestRunFinished(e);

  46:     }

  47: }

2. Next you want to update the DriverPage.aspx to use our new LogTestcaseExecutor. We’ll override the RunTestcases method and set the TestcaseExecutor property to our custom one. (Notice that  I pass the path to where the log is going to get written).

   1: public partial class DriverPage : TestDriverPage

   2: {

   3:     protected override void RunTestCases()

   4:     {

   5:         this.TestcaseExecutor = new LogTestcaseExecutor(Server.MapPath(""));

   6:         base.RunTestCases();

   7:     }

   8:  

   9: }

That’s it. As an example, after running some tests the log will be under the Test directory.

And this is how the custom log looks like (yes, not fancy, but its the idea that matters :P)

You can use this feature to log testcase execution in anyway that you see fit. 

Federico Silva Armas
SDET ASP.NET QA

Download : Episode 1

In the first ever ASP.NET QA Podcast Federico and Matthew discuss the motivation behind the community involvement from the QA team and the release of the QA team’s Lightweight Test Automation Framework for ASP.NET and what it is.

• Introductions to the podcasters, Federico and Matthew.
• What is it that the ASP.NET QA team does.
• The recent release of ASP.NET MVC RC2.
• The recent release of the source code to the Lightweight Test Automation Framework.
• The purpose of the release of the framework to codeplex.com and where it fits in the market.
• Why the QA team chose to write their own automation framework.

Links from the show :

• ASP.NET MVC RC2 Release
• Federico’s Email
• Matthew’s Email
• Lightweight Test Automation Framework forum
• Download The Lightweight Test Automation Framework

Disclaimer : This is our first try at creating, editing, and publishing a podcast.  Please bear with us as we iron out all the wrinkles in the process. Your feedback is greatly appreciated.

 This post is meant to illustrate some capabilities of the Lightweight Test Automation Framework.

Suppose I want to create a small application that displays the latest posts that where made to our forum: http://forums.asp.net/1193.aspx. I would like to issue a WebRequest to the forum and parse the HTML and find the titles of all the posts in the main page. There are probably lots of libraries to parse HTML content, but I'll show how you can use our framework to accomplish this.

1. The first thing to do is to become familiar with the HTML page that you want to parse, in this case I navigate to the forum and using a DOM inspector I can see that all the links to the posts are inside table rows that have the class attribute set to "CommonListRow"

2. Next, I created a new Console application and reference the Microsoft.Web.Testing.Light.dll

3. Make a request to the server (in my example, I use the System.Net.WebRequest class).

4. Use the static HtmlElement.Create(string html) to parse the response into an HtmlElement.

5. Use the common API to find the elements that you need.

Here is the source code:

        static void Main(string[] args)
        {
            // Create a request for the URL.         
            WebRequest request = WebRequest.Create("http://forums.asp.net/1193.aspx");
           
            // Get the response.
            HttpWebResponse response = (HttpWebResponse)request.GetResponse();

            // Get the stream containing content returned by the server.
            Stream dataStream = response.GetResponseStream();

            // Open the stream using a StreamReader for easy access.
            StreamReader reader = new StreamReader(dataStream);
            
            // Read the content.
            string responseFromServer = reader.ReadToEnd();

            // Remove the DOCTYPE
            responseFromServer = Regex.Replace(responseFromServer, @"\<\!DOCTYPE.*?\>", String.Empty);       
     
            // Load the response into an HtmlElement
            HtmlElement rootElement = HtmlElement.Create(responseFromServer);

            // find all the post rows
            HtmlElementFindParams findParams = new HtmlElementFindParams();
            findParams.TagName = "tr";
            findParams.Attributes.Add("class", "CommonListRow");
            foreach (HtmlElement tableRow in rootElement.ChildElements.FindAll(findParams))
            {
                //find the first link within the row
                HtmlAnchorElement link = (HtmlAnchorElement) tableRow.ChildElements.Find("a", 0);

                // Display the title
                Console.WriteLine(String.Format("\"{0}\"",link.CachedInnerText));

                //Display the link
                Console.WriteLine(String.Format("\thttp://forums.asp.net{0}\n", 
                      link.CachedAttributes.HRef));
            }

            // Cleanup the streams and the response.
            reader.Close();
            dataStream.Close();
            response.Close();

        }

The Asp.NET QA team has been working to make what we do and our processes more visible to our customers.  Two major things we have done to increase this are, releasing the Lightweight Test Automation Framework binaries with a sample to Codeplex.com and starting our ASP.NET QA Blog.  Well, over the last week we reached another milestone by releasing the source to the Lightweight Test Automation Framework on Codeplex.com.  This is the first time we have tried to do something like this on the ASP.NET QA team and we are very excited.  Visit Codeplex.com and check it out for yourself!

Items Updated in the February Update of the Lightweight Test Automation Framework include

• Lightweight Test Automation Framework road map
• Lightweight Test Automation Framework release notes
• Lightweight Test Automation Framework for ASP.NET Samples
• Lightweight Test Automation Framework for ASP.NET Source

Thanks go out to all those who have helped to get this release out the door, with a special thanks to the lead developer Federico Silva Armas.