Performance testing with 10,000 concurrent users
Lately, I have been concentrating on performance testing within an application I am working on. This is using the Team test portion of Visual Studio. It certainly is a powerful tool and is exponentially better than anything offered to date by Microsoft.
We have some requirements to ensure we hit around 5,000 concurrent users within this app, so pumping to 10,000 is a good way of ensuring that we can meet the goals and find any potential bottlenecks or issues.
Currently, I have all the test runs automated so that every night, a 10,000 concurrent user test is initiated that runs for approximately 12 hours. On a weekend, the automated test run, also tests using 10,000 concurrent users, but runs for approximaely 60 hours. The weekend test is identical to the overnight run except that it lasts longer, and has a slower ramp up to the 10,000 users.
Additionally, we have a web farm of 7 web servers, all connecting to a database server, and all these servers have a set of performance counters that are also scheduled to start and stop at the times of each test. These results get logged to a database, so that I can look through them the next day.
We are using a client machine farm of approximately 20 PC's all administered remotely through the Team test console, to generate the load of 10,000 concurrent users. Sometimes we use more if the machines are available. This is one of the cool aspects of team test, I can install the load agent on a large amount of machines and enlist only some of those machines by manually, or by specifying properties of each machine and only enlist the machines that contain a particular property or attribute.
I will digress into the details of the test rig setup and how I go about setting up the automation and collection of performance data on later posts. This is simply an initial post to paint the picture on what I am doing at the moment.
I will say the the ability to examine the test result data using the interactive graph capability within team test is awesome. The user interface is a little quirky and not entirely intuitive at times, but you get used to it.