News from the QA team
Sunday, September 13, 2009 11:40 PM

I know there has been very little traffic on our team blog, but I recently decided to start my own blog so that I could share my personal opinions. However, here is a small news update from the ASP.NET QA team:

  • First and fore most: the ASP.NET QA team is hiring! If you are interested, check out our job posting here. You can also read about it in Matthew Osborn’s blog here.
  • The weekly CodingQA podcast is alive and kicking. Check it out, and if you like it you can subscribe using iTunes and Zune.
  • Jim Wang, the lead tester for ASP.NE Ajax, posted a nice tutorial using ASP.NET Ajax Preview 5 here.
  • Federico Silva Armas (me :P) is presenting at DevConnections this year. Check this post for more information.
  • Last but not least, Tales of Testing is my personal blog where I cover everything that has to do with testing. If you are into testing and want to share your ideas I am interested in listening.

 

Federico Silva Armas
SDET, ASP.NET QA Team

CodingQA Podcast Episodes 10-12
Friday, July 03, 2009 10:01 AM

Episode 10 Project Structure

In this show Matthew and Federico talk about how the ASP.NET team structures it’s sources and the theory behind it all. They talk about how the team handles the organization of all of it’s sources and what seems to work well along with what seems to cause hang ups.

Episode 11 Interviewing Hong Li

In the eleventh installment of Coding QA Federico is on vacation so Matthew sits down with his mentor, Hong Li, for an interview. Join Matthew as he learns a little about one of his fellow team members, servicing an released product, and working with team members that are offshore in China.

Episode 12 Jim Wang on Microsoft AJAX

In the twelfth installment of Coding QA Federico is still enjoying his vacation so Matthew sits down with his officemate Jim Wang to discuss the ins and outs of test the Microsoft AJAX libraries. Tune in as Matthew picks Jim’s brain on what it was like to test a client side framework.

 

Federico Silva Armas
SDET, ASP.NET QA Team

by farmas | with no comments
Filed under: ,
How to write a custom log in LTAF June release?
Wednesday, June 10, 2009 3:35 PM

If you want to write a custom log while running your test, the June 2009 release of LTAF now provides an easier way to set that up. The DriverPage gives access to a TestcaseExecutor object that exposes several lifecycle events

Here is a code snippet that shows how to write your own log (MyCustomLog.txt) by attaching to the proper events to gather information while the tests are executing.

private StringBuilder _log;
private string _logPath;

protected override void RunTestCases()
{
    // if user checked the Write Log To Disk checkbox
    if (this.WriteLogToDiskCheckBox.Checked)
    {
        // attach event handlers to write my custom log
        this.TestcaseExecutor.TestcaseExecuted += 
            new EventHandler<TestcaseExecutionEventArgs>(TestcaseExecutor_TestcaseExecuted);
        
        this.TestcaseExecutor.TestRunFinished += 
            new EventHandler<TestRunFinishedEventArgs>(TestcaseExecutor_TestRunFinished);

        this.TestcaseExecutor.TestRunStarted += new EventHandler(TestcaseExecutor_TestRunStarted);

        _logPath = System.IO.Path.Combine(Server.MapPath(""), "MyCustomLog.txt");
        _log = new StringBuilder();
    }
    
    base.RunTestCases();
}

void TestcaseExecutor_TestRunStarted(object sender, EventArgs e)
{
    _log.AppendLine("Test Run Started.");
}

void TestcaseExecutor_TestcaseExecuted(object sender, TestcaseExecutionEventArgs e)
{
    _log.AppendLine(String.Format(
        "Finished running test '{0}'. Status='{1}'", 
        e.WebTestName, 
        e.Passed));
}

void TestcaseExecutor_TestRunFinished(object sender, TestRunFinishedEventArgs e)
{   
    _log.AppendLine("Test Run Finished.");
    
    // make sure that you have write access to the test folder
    System.IO.File.WriteAllText(_logPath, _log.ToString());
}

The "MyCustomLog.txt" file will be written into the same folder where the DriverPage.aspx exists.

Federico Silva Armas
SDET, ASP.NET QA Team

by farmas | with no comments
Filed under: ,
What's new in LTAF June release?
Wednesday, June 10, 2009 3:34 PM

The June 2009 release of the Lightweight Test Automation Framework is out on CodePlex. Here is a small write up of the new features available.

Automation of frames

Now tests can automate pages that contain framesets and iFrames. For example:

[WebTestMethod]
    public void Frame()
    {
        HtmlPage page = new HtmlPage("TestFrameSet.htm");

        // get top frame
        HtmlPage frame = page.GetFramePage("topFrame");

        // verify title of frame
        Assert.AreEqual("This is the top frame", frame.Elements.Find("h1", 0).GetInnerText());
    }

 

Write to the test console

This is an experimental feature that is useful to debug tests. You can now write messages from your test into the test console that appears in the driver page. For example:

[WebTestMethod]
    public void PopupPageWithIndex()
    {
        // Navigate to the frameset
        HtmlPage page = new HtmlPage("TestFrameSet.htm");

        //get the top frame
        WebTestConsole.Write("Get the top frame");
        HtmlPage frame = page.GetFramePage("topFrame");

        // click on the button that opens a new page
        WebTestConsole.Write("Click the openwindow button");
        frame.Elements.Find("OpenWindow").Click();

    ...

When run, this test will output traces into the console UI:

image

 

Hide the test console

New option in the UI allows user to hide the test console. This is useful for long running test suites, to prevent the browser from maintain a grown list of commands.

image

 

Quickly verify page is rendering an asp.net error

New helper method in HtmlPage makes it easy to quickly verify if a asp.net error page is being rendered. This is useful to immediately stop the test if an error has occurred, instead of waiting for a timeout. For example:

[WebTestMethod]
    public void VerifyTheAspNetErrorPage()
    {
        HtmlPage p = new HtmlPage("PageWithError.aspx");
        bool pageFail = p.IsServerError();
        Assert.IsFalse(pageFail);

        ...
    }

 

Exposed test run lifecycle events

Users can now register handlers in the DriverPage.aspx to several life cycle events. This is useful, for example, to generate a custom log:

protected override void RunTestCases()
   {
       // if user checked the Write Log To Disk checkbox
       if (this.WriteLogToDiskCheckBox.Checked)
       {
           // attach event handlers to write my custom log
           this.TestcaseExecutor.TestcaseExecuted += 
               new EventHandler<TestcaseExecutionEventArgs>(TestcaseExecutor_TestcaseExecuted);
           
           this.TestcaseExecutor.TestRunFinished += 
               new EventHandler<TestRunFinishedEventArgs>(TestcaseExecutor_TestRunFinished);    
       }
       
       base.RunTestCases();
   }

 

Additional Resources

 

Federico Silva Armas
SDET, ASP.NET QA Team

by farmas | 2 comment(s)
Filed under: ,
CodingQA Podcast Episode 9
Saturday, June 06, 2009 12:09 PM

Download: Episode 9
Streaming: Episode 9

In this show Matthew and Federico talk about what the ASP.NET QA team does to prepare for a release. Taking into account the recent release of ASP.NET 4 Beta 1, they’ll go over several of the "exit criteria" that the product must meet before delivery to the public.

News

  • Lightweight Test Automation Framework June Release

Sign off on a release

  • Difference between Quality Gates and Exit Criteria
  • Accessibility. Mainly manual tests to validate feature conforms to accessibility standards.
  • Code Coverage. Metric used to identify problematic development areas.
  • Stress. Specialized automated long running tests to spot memory leaks and other stress problems.
  • Localization. Verification that the product can be correctly localized to several languages.
  • Test Passes. Functional verification of the product across many platform combinations.
  • Zero Active Bugs. Series of step down goals to drive feature crews to finish development.
by farmas | with no comments
Filed under: ,
CodingQA Podcast Episode 8
Saturday, June 06, 2009 12:06 PM

Download: Episode 8
Streaming: Episode 8

This show is an open discussion about what testing means. Federico and Matthew share their opinions around the meaning of "testing", "quality" and the role of a QA team. They don't always agree but each brings something unique to the table on topics so basic yet somewhat mysterious in the testing discipline.

News

Topics of Discussion

  • What is the role of QA?
  • What is testing?
  • What is quality?
  • Should quality be measurable?
  • What is risk?
  • What is the difference between Dev and Test?
by farmas | 3 comment(s)
Filed under: ,
CodingQA Podcast Episode 7
Monday, May 25, 2009 11:28 PM

Download: Episode 7
Streaming: Episode 7

In this show Federico and Matthew talk about what exploratory testing (ET) is and how the ASP.NET QA team uses this methodology for testing.

News

Exploratory Testing

  • Definition
  • Differences between exploratory testing and scripted testing
  • Differences between exploratory testing and ad-hoc testing
  • The 4 principles of exploratory testing
    • Creativity
    • Goal oriented
    • Reproducibility
    • Adaptability
  • How the ASP.NET QA team uses exploratory testing
  • When is exploratory testing a good fit to test a product
  • Brief example of an exploratory testing session
  • What are the outcomes of an exploratory testing session
    • Scenarios tried
    • Questions
    • Assumptions
    • Defects
    • Future scenarios
by farmas | with no comments
Filed under: ,
CodingQA Podcast Episode 6
Thursday, May 21, 2009 11:29 PM

Download: Episode 6
Streaming: Episode 6

In this show Federico and Matthew explore the question of "when too much automation is too much?" Based on their experiences in the ASP.NET QA team, they talk about the danger of automating for the sake of automation, the problems that it can create, and the value of context driven testing.

News

  • New name and name website: Coding QA.
  • .NET Micro Framework has been open sourced.

Automation for the sake of automation

  • What is the goal of automation? To find bugs during development vs. to find regressions after development.
  • What problems can arise from having too much automation? The tale of the ASP.NET QA "boat anchor".
  • Context matters when deciding how to ensure a feature does not regress.
  • What parameters can be used to decide the testing approach
    • Automation cost.
    • Time of delivery.
    • Resources available.
    • Risk of a regression.
    • Customer impact if a regression is introduced.
    • Quickness between when the regression is introduced and when it is discovered.
  • Real examples from the team
    • Testing event handler generation for web controls in VS.
    • Testing picker dialogs for web controls in VS.

 

Federico Silva Armas
SDET, ASP.NET QA Team

by farmas | with no comments
Filed under: ,
CodingQA Podcast - Episode 5
Sunday, May 17, 2009 1:19 AM

Download: Episode 5
Streaming: Episode 5

In this episode Matthew interviews Carl Dacosta, lead tester for the MVC 1.0 project, and talks about what it was like to develop and test a product with so many releases.

  • MVC crossed the 100,000 downloads mark (currently at 140,000)
  • MVC released under MS-PL license
  • Interview with Carl Dacosta
    • Some history on Carl and how he got into test.
    • Role of Carl as a tester in the MVC crew.
    • What was the development and testing process of MVC to enable frequent releases.
    • Parts of the testing process that worked well for this kind of project, and parts that didn't work so well.
    • How to handle quality of releases when external partners are starting to rely on the framework.
    • What did Carl enjoy the most working on the MVC crew.
    • Testing the "unit testing" experience for developers working on MVC.

 

Federico Silva Armas
SDET, ASP.NET QA Team

by farmas | with no comments
Filed under: ,
CodingQA Podcast Feed
Sunday, May 17, 2009 1:17 AM

We now have a proper feed for our podcast, head over to codingqa.com to subscribe. You can also find us in iTunes and very soon in Zune Marketplace as well. 

For the time being I’ll keep posting here whenever an episode comes out. .

Federico Silva Armas
SDET, ASP.NE QA Team

by farmas | with no comments
Filed under: ,
More Posts Next page »