WCF stress testing using BizTalk Server LoadGen
Stress testing is one of key aspects that developers should take into consideration in order to maximize the performance of Web Service deployments. Windows Communication Foundation (WCF) provides a very extensible framework for monitoring and configuring WCF Services performance profiles. However one key pieces of successful load testing is precisely to execute the correct tests and for that using the correct tools is fundamental. Load tests are typically based on relatively complex algorithms that encompass aspects such as thread management, memory, message throughput, etc. For instance, a typical Web Service test might include sending a thousand messages per second to a Web Service using five threads until the consumed memory hits 1GB. Such algorithms require a detailed knowledge of the Web Service technology as well as the operating system being used.
There are a variety of tools and frameworks to execute load tests algorithms against Web Services. MindReef's SOAPScope remains my favorite (even though my buddy Tim Ewald is not part of the team anymore J) although I always like to consider alternatives especially for Microsoft CS (not J2SE friendlyJ) technologies. Stress testing becomes even more challenging with sophisticated Web Services toolkits such as WCF or Apache Axis2 given that most load tests tools don't support advanced Web Services features such as WS-* protocols or multi-transport and in the few cases they do interoperability becomes a big challenge. Consequently, in order to test those complex Web Services, developers need to base the tests on an interoperable Web service technology which brings me to the topic of this post J.
Lately I've spent some time playing with my favorite BizTalk Server load generation tool (LoadGen) to stress test WCF Services. Until now LoadGen has been mainly considered a BizTalk centric tool with the capability to execute complex load testing algorithms against a variety of adapters and measuring BizTalk specific counters. However the latest version of LoadGen adds support for the WCF adapters included in BizTalk Server R2 which also makes it ideal for testing plain WCF Services J. Let's take the following service as an example.
|
[ServiceContract()] public interface IMyService { [OperationContract(Action="Action")] int Add(int param1, int param2); }
public class MyService : IMyService { public int Add(int param1, int param2) { return param1 + param2; } } |
In order to test that service using LoadGen we can use the following message that will be embedded in the SOAP body.
|
<Add xmlns="http://tempuri.org/"> <param1>11</param1> <param2>11</param2> </Add> |
Now we can follow the traditional techniques to configure LoadGen. The following configuration file instructs LoadGen to send 500 messages using four threads 50 to our service in a 20 seconds interval.
|
<LoadGenFramework> <CommonSection> <LoadGenVersion>2</LoadGenVersion> <OptimizeLimitFileSize>8192</OptimizeLimitFileSize> <NumThreadsPerSection>4</NumThreadsPerSection> <SleepInterval>5</SleepInterval> <LotSizePerInterval>10</LotSizePerInterval> <RetryInterval>10000</RetryInterval> <ShutdownDelay>10000</ShutdownDelay>
<ThrottleController Mode="Disabled">
<Monitor Name="Rate"> <Assembly>RateMonitor.dll/RateMonitor.RateMonitorLib</Assembly> <ThresholdRange>50-50</ThresholdRange> <SleepInterval>5</SleepInterval> <Parameters></Parameters> </Monitor>
<ThrottleCondition>Rate</ThrottleCondition>
</ThrottleController>
<StopMode Mode="Files"> <NumFiles>500</NumFiles> <TotalTime>20</TotalTime> </StopMode>
<Transport Name="WCF"> <Assembly>WcfTransport.dll/WcfTransport.WcfTransport</Assembly> </Transport> </CommonSection>
<Section Name="WcfSection"> <SrcFilePath>addmsg.xml</SrcFilePath> <DstLocation> <Parameters> <ClientType>WcfTwoWaysVoid</ClientType> <Transactional>false</Transactional>
<ClientEndPointName >BasicHttpBinding_IMyService</ClientEndPointName> <IsSynchronous>false</IsSynchronous> <MessageAutoGenerate>false</MessageAutoGenerate> <LogReqMessage>false</LogReqMessage> <LogRespMessage>false</LogRespMessage> <DisplayHelperMsgsOnConsole>false</DisplayHelperMsgsOnConsole> <MaxConcurrentMessages>200</MaxConcurrentMessages> </Parameters> </DstLocation> </Section> </LoadGenFramework> |
One of the things that you might have noticed in the load configuration file is that there are no references to the target WCF service. We can specify those settings in the LoadGenConsole.exe.config file as illustrated in the following figure.
|
<?xml version="1.0" ?> <configuration>
LoadGen configuration settings....
<system.serviceModel> <extensions> <behaviorExtensions> <add name="loadgenwcfinterceptor" type="WCFTRansport.ActionModifierBehavior.LoadGenBehaviorBindingElement, WCFTRansport.ActionModifierBehavior, Version=1.0.0.0, Culture=neutral, PublicKeyToken=79de3c67dc9393f3"/> </behaviorExtensions> </extensions>
<bindings> <basicHttpBinding> <binding name="BasicHttpBinding_IMyService"> <security mode="None"> <transport clientCredentialType="None" proxyCredentialType="None" realm="" /> <message clientCredentialType="UserName" algorithmSuite="Default" /> </security> </binding> </basicHttpBinding> </bindings>
<client> <endpoint address="http://localhost:8080/WCF/Service.svc" binding="basicHttpBinding" bindingConfiguration="BasicHttpBinding_IMyService" contract="WcfTransport.IChannelAsyncNonTransactional" name="BasicHttpBinding_IMyService" /> </client> </system.serviceModel>
</configuration> |
From the WCF perspective LoadGen is just another client and we can take advantage of all the client configuration options available with WCF.
In order to run the previous configured test we just need to use the LoadGen console passing the test configuration as an argument. The following figure shows a snapshot of the test results displayed in the LoadGen console.
What about performance monitoring?
On the service side we can now use monitoring tool to measure the performance of the server. There are a variety of alternatives to achieve this, the simplest one might be a combination of the WCF native trace listeners and performance counters. The following figure shows a snapshot of my performance monitor while I was executing the previous test.
More complex alternative might involve the use of sophisticated monitoring tools such as System Center, HP Open View, etc.
Support for multiple bindings
The service of our previous example was hosted in an endpoint using the basicHttpBinding WCF binding which makes it highly interoperable from a consumer perspective. However there are a lot of WCF services that use more complex binding to address more sophisticated scenarios. As we explained in the previous section, LoadGen allows to automatically plugging in a client binding configuration on the LoadGenConsole.exe.config which makes it no different from any other WCF client.
Using these feature developers can test any WCF service as long as the policy requirements can be addressed via WCF configurations. Additionally, this means that developers can take full advantage of WCF client side extensibility mechanisms such as custom client behaviors, message-parameter inspectors, binding elements, channels to extend the default capabilities of LoadGen.