ISerializable - Roy Osherove's Blog

I've been using Application Center Test (ACT) and Web Application Stress (WAS) tool. I'm not sure on the comparison of features between these tools and what you're using are.

Do you have any thoughts as to their comparisons?

Thanks! That's a great list. There's also http://nunitasp.sourceforge.net/ - it's an extension of NUnit for ASP.NET.

Doug:S orry - I have'nt had any experiece with the tools you mentioned. Web dev is a very new subject for me, so my guess is you can probably help me on this subject much more than I can help you :) However, If there's one dude I kknow *can* answer that question it's Addy Santo (http://weblogs.asp.net/asanto) . Maybe he'll actually read this comment and answer here...

Hi guys! Thanks for the props Roy - now I really don't have a choice about answering, do I ;)

I don't regard WAS as a GUI testing tool. WAS reports data which is mainly performance oriented (ttfb,ttlb, page sizes, std.dev, etc) and the WAS scripts are targeted at perf testing scenarios. Things like conditional branches are really non-trivial.

ACT is basically an updated version of WAS. The limitations are the same.

For real UI testing you should probably go with a professional testing tool - my favorite is the Mercury toolset, but there also quite a few others out there.

-Addy

Just a bit more info regarding WAS/ACT:

They set themselves up as an HTTP proxy between your browser and the test site, and log the HTTP requests which your browser generates while you use the site. You can then "replay" the logged HTTP request stream to the server to simulate multiple users. The server's response is then analysed and displayed in a friendly report. The report includes mainly perf related data. Other than the page size in bytes and error counts,there isn't enough info in there to actually determine if the app behaves as expected.

There is some parameterization available- for instance you can simulate multiple users - but not enough for real UI testing.

I recently started using NUnitASP. The learning curve wasn't that steep, and the results have been really good. My unit tests insert data on the pages, then verify results in the database. It can follow links, push buttons and so on.

It doesn't cover all cases, however - support will surely be added for certain controls as the app evolves - but I did come up with a small DLL that lets me parse the HTML the server gives me. I can pretty much find whatever I need to on the page with one or two lines of code, even when the control isn't supported by NUnitASP.

I also tried XmlTestSuite (http://xmltestsuite.sourceforge.org), but found it lacking. Has anyone had luck making that work with .NET?

I think some people are a bit confused about testing. Be aware that "When holding a hammer, everything tends to look like a nail" - UI testing and unit testing are very different and each has its own goals and methodologies.

Unit tests are designed to ensure that each system component behaves per spec in an isolated environment, and that there are no bugs, side-effects or dependencies. After unit testing all the components, any remaining bugs should be related to integration issues only. Unit tests are usually very technical tests which are prepared by the developers who wrote the components. The blueprints for unit test planning arethe detailed design document, which are written by the architecture/design teams.

UI Testing is meant to validate that the interaction between the app and the end-user is per spec. The emphasis is more on completing processes and scenarios, and the testing is done by people who are not necessarily developers. The blueprints for UI testing are the requirements and/or UI designs which are authored by the biz analysts or UI designers.

The only case I can think of where UI and Unit tests could actually meet are when a system includes components which include a simple, self-contained UI. Example: widgets such as "Load File" dialog.




UI testing usually tests more involved scenarios and longer processes than unit testing

Andy, great point on the different types of testing.

FYI, I use ACT to simulate (script) UI interactions to make sure that I get the correct response based on the inputs delivered (unit testing). I keep the script so I can run the same unit tests as the pages/modules get updated.

ACT isn't really good for stress testing because it can't really simulate different IPs hitting your site.

FOr small shops I recommend ACT/WAS for unit testing because their apps are smallish and the testing can be done pretty well with the tools delivered as part of VS.NET. For enterprise development, I agree that an enterprise tool like NUnit, ANTS, SoftICE< or the QA-Series (from CompuWare) are the better choices.