March 2009 - Posts

ASP.NET QA Podcast – Episode 3
Tuesday, March 31, 2009 4:07 PM

Download : Episode 3

In the third installment of the ASP.NET QA Podcast Federico and Matthew interview Mark Berryman, a test lead on the ASP.NET QA Team.  Join them as they learn the ins and outs of test management through Mark’s eyes.

  • Release of the Mobile Browser Database by Live Dublin.
  • April refresh of the Lightweight Test Automation Framework.
  • Interview with Mark Berryman.
    • What is the difference between Test Manager and Test Lead.
    • A little history of Mark and how he came to join the ASP.NET QA team.
    • Responsibilities of a Test Lead / What Mark is working on.
    • What Mark looks for in a tester on his team.
    • Challenges Mark faced when he was an IC (individual Contributor).
    • What Mark would change on items he has tested in the past.
    • How Mark Manages risk on his team.
    • Who is Marks favorite person in the whole world.

Links from the show :

by osbornm | 2 comment(s)
Filed under: , ,
ASP.NET QA Podcast – Episode 2
Friday, March 20, 2009 11:51 AM

Download : Episode 2

In the second installment of the ASP.NET QA Podcast Federico and Matthew discuss MIX 09 announcements and the ins and outs of supporting various browser versions on the ASP.NET QA team.

  • MIX 09 Conference announcements
    • MVC RTM 1.0
    • Silverlight 3.0
    • Web Application Installer
    • Web Application Gallery
  • What browsers the team tests on and how they decided which browsers to test.
  • What it means when the team says they support a browser.
  • How the team manages the risk of not testing on every possible browser.
  • The timeline for testing features on multiple browsers.
  • The infrastructure the team has to test on multiple browsers.
  • How the team keeps up with the fast growing browser market, such as the release of four major browsers in just a few months.

Links from the show :

by osbornm | 1 comment(s)
Filed under: , ,
How to test Asp.Net ControlDesigners without VisualStudio? (PROTOTYPE)
Friday, March 20, 2009 12:22 AM

I was recently tasked to prototype a way to test our ControlDesigners without using VisualStudio. For those of you that are not familiar with ControlDesigners here is an overview. In short, ControlDesigners is what give an Asp.Net runtime control a design time experience when hosted inside VisualStudio. Traditionally, our team (and most teams in DevDiv) prefer to automate all of VisualStudio to get regression coverage of designer features.

Now, ControlDesigners that were written for Asp.Net 1.0 and 2.0 do not have unit tests (yeah, our team didn't see the light until later), so our QA team has had to maintain a big automation bed to verify regressions in our ControlDesigners that is generally a pain to write (constant product and framework changes, coupled with the fact that the Visual Web Developer team rewrote the design surface twice make VisualStudio UI testing very challenging). So one approach that we are investigating is to see if we can add unit tests for our old ControlDesigners.

You can download a simple prototype of how such testing may look like HERE.

This is very early, but it gave us good confidence that we could probably test most of our core controls. Here is an example of how a very simple test looks like:

   1: [TestMethod]
   2: public void VerifyRenderingWithId()
   3: {
   4:     TestDesigner designer = new TestDesigner();
   5:     HyperLink control = new HyperLink();
   6:     control.ID = "HyperLink1";
   7:  
   8:     designer.VerifyRendering(@"<a id=""HyperLink1"" Href=""url"">[HyperLink1]</a>", control);
   9: }

 

There is also a more advanced test to verify the design time rendering of  a GridView control that is bound to a DataSource control: 

   1: [TestMethod]
   2: public void VerifyRenderingWithSchema()
   3: {
   4:     TestDesigner designer = new TestDesigner();
   5:  
   6:     TestDataSourceViewSchema schema = new TestDataSourceViewSchema();
   7:     schema.AddField(new TestDataSourceFieldSchema());   
   8:     schema.AddField(new TestDataSourceFieldSchema());
   9:     designer.AddDataSourceControl("DataSource1", schema);
  10:  
  11:     GridView control = new GridView();
  12:     control.ID = "GridView1";
  13:  
  14:     ControlDesigner controlDesigner = designer.AddDataBoundControl(control, "DataSource1");
  15:  
  16:     string expectedHtml = @"<table cellspacing=""0"" rules=""all"" border=""1"" id=""GridView1"" style=""border-collapse:collapse;"">
  17:         ... 
  18:         cut for space
  19:         ...
  20:         ";
  21:     designer.VerifyRendering(expectedHtml, controlDesigner);
  22: }

 

Some comments regarding this project:

  • The idea is that most ControlDesigners make calls into our runtime controls, which makes the designer vulnerable to changes to our runtime code. Having this level of simple regression tests can help us catch changes that could break the design time experience of controls.

  • Notice that I am making simple string comparisons to verify big chunks of HTML code. Internally, we use a TagDiffer tool that helps in abstracting common rendering changes that should not break the test (like attribute ordering). I had to modify the project to not include these binaries since I am not sure that I am allowed to release it.

  • I did a little bit of research to see if I could find a mock implementation of the designer that would help me with this. I could not find anything, but if anybody knows of a tool that we can use or has some pointers of something similar I would really appreciate it.

  • There are many services that are not implemented, so plugging other controls into this may or may not work :P

Any feedback you have is welcome.

Federico Silva Armas
SDET ASP.NET QA Team

by farmas | 1 comment(s)
Filed under: ,
Lightweight Automation Framework: How to create your own log
Thursday, March 19, 2009 1:33 AM

 Update: With the newest release of LTAF there is a simplified way of attaching to events that can be used to generate a custom log. Read more about it in this post.

One question that I got from those using the Lightweight Test Automation Framework is how to create your own log. In this post I’ll go over the changes you need to make to create your own log that outputs an html table with the tests that ran and their results. The output will be really simple, but it is meant as a sample.

1. The first thing is to create a class that inherits from TestcaseExecutor. This class lets you override methods that are called when each test case starts and ends, and we’ll use it to create our custom log. We’ll also override a method that is called at the end of all test run, where we will write the log to disk (notice that in this sample asp.net needs write access to the Test directory)

   1: public class LogTestcaseExecutor: TestcaseExecutor
   2: {
   3:     private readonly string _testPath;
   4:     private StringBuilder _log;
   5:  
   6:     public LogTestcaseExecutor(string testPath)
   7:         : base()
   8:     {
   9:         _testPath = testPath;
  10:         _log = new StringBuilder();
  11:     }
  12:  
  13:     protected override void OnTestcaseExecuted(TestcaseExecutionEventArgs e)
  14:     {
  15:         string tableRow = @"
  16: <tr style='"background-color:{2};"">
  17:     <td>{0}</td>
  18:     <td>{1}</td>
  19: </tr>
  20: ";
  21:         _log.AppendFormat(tableRow,
  22:             e.WebTestName,
  23:             (e.Exception == null) ? "&nbsp" : e.Exception.Message,
  24:             (e.Passed) ? "green" : "red");  
  25:         
  26:         base.OnTestcaseExecuted(e);
  27:     }
  28:  
  29:     protected override void OnTestRunFinished(TestRunFinishedEventArgs e)
  30:     {
  31:         string log = @"
  32: <html>
  33:     <body>
  34:         <table style='"width:100%;border-style: solid;border-width: 1px;"">
  35:             <tr>
  36:                 <td>Testcase</td><td>Error</td>
  37:             </tr>
  38:             {0}
  39:         </table>
  40:     </body>
  41: </html>";
  42:  
  43:         File.WriteAllText(Path.Combine(_testPath, "RunLog.htm"), String.Format(log, _log.ToString()));
  44:         
  45:         base.OnTestRunFinished(e);
  46:     }
  47: }

 

 

2. Next you want to update the DriverPage.aspx to use our new LogTestcaseExecutor. We’ll override the RunTestcases method and set the TestcaseExecutor property to our custom one. (Notice that  I pass the path to where the log is going to get written).

   1: public partial class DriverPage : TestDriverPage
   2: {
   3:     protected override void RunTestCases()
   4:     {
   5:         this.TestcaseExecutor = new LogTestcaseExecutor(Server.MapPath(""));
   6:         base.RunTestCases();
   7:     }
   8:  
   9: }

 

That’s it. As an example, after running some tests the log will be under the Test directory.

image

 

And this is how the custom log looks like (yes, not fancy, but its the idea that matters :P)

image

You can use this feature to log testcase execution in anyway that you see fit. 

Federico Silva Armas
SDET ASP.NET QA

by farmas | with no comments
Filed under: , ,
ASP.NET QA Podcast – Episode 1
Wednesday, March 11, 2009 9:25 PM

Download : Episode 1

In the first ever ASP.NET QA Podcast Federico and Matthew discuss the motivation behind the community involvement from the QA team and the release of the QA team’s Lightweight Test Automation Framework for ASP.NET and what it is.

  • Introductions to the podcasters, Federico and Matthew.
  • What is it that the ASP.NET QA team does.
  • The recent release of ASP.NET MVC RC2.
  • The recent release of the source code to the Lightweight Test Automation Framework.
  • The purpose of the release of the framework to codeplex.com and where it fits in the market.
  • Why the QA team chose to write their own automation framework.

Links from the show :

Disclaimer : This is our first try at creating, editing, and publishing a podcast.  Please bear with us as we iron out all the wrinkles in the process. Your feedback is greatly appreciated.

by osbornm | 1 comment(s)
Filed under: , , ,
More Posts