How fast are my services? Comparing basicHttpBinding and ws2007HttpBinding using the SO-Aware Test Workbench

When working on real world WCF solutions, we become pretty aware of the performance implications of the binding and behavior configuration of WCF services. However, whether it’s a known fact the different binding and behavior configurations have direct reflections on the performance of WCF services, developers often struggle to figure out the real performance behavior of the services. We can attribute this to the lack of tools for correctly testing the performance characteristics of WCF services under different load profiles. As sad as this sound, the fact of the matter is that the existing web service load test tools in the market can barely test anything other than the basicHttpBinding.

With the recent release of the SO-Aware Test Workbench, we attempted to provide a simple solution to the load and performance testing of WCF/Web Services. SO-Aware Test Workbench enables developers to model and execute load tests with only a couple of clicks.

Let’s take a real example:

BasicHttpBinding and ws2007HttpBinding are, undoubtedly, the two most common bindings in WCF solutions in the real world. At some point, we’ve all heard that the ws2007HttpBinding can slightly impact the performance of a service given that, on its default configuration, it leverages WS-SecureConversation to secure the communication between client and service. However, I’ve never seen a good benchmark that details how real is the performance impact of using the ws2007HttpBinding.

Time to find the answer!!!

Using the SO-Aware Test Workbench, we can run a series of load tests against services configured with the basicHttpBinding and ws2007HttpBinding and compare the results. For the purpose of this sample, let’s use the following WCF service.

basicHttpBinding vs. ws2007HttpBinding: Simple Test Strategy

The simple test strategy simulates a fixed concurrent number of clients during a specified period of time.

When we run a simple load profile against that simulates our sample client we obtain the following results. Please notice the low average response time and large number of test generated as well as the high thread count which indicates the high performance of the service.

Running the same load profile against the service configured using the ws2007HttpBinding produces the results highlighted in the following figure. Notice that there the response time are a bit higher and the number of threads used a bit lower which clearly illustrates the performance impact introduced by the use of the ws2007HttpBinding.

basicHttpBinding vs. ws2007HttpBinding: Linear Test Strategy

The linear strategy is small variation of the simple strategy on which the concurrent number of clients fluctuates between two limits. When our service is configured with the basicHttpBinding, it exhibits the following behavior when running a linear strategy simulating between 100 to 125 concurrent clients.

The same test ran against the service using the ws2007HttpBinding produces the following results.

Again, notice the slight performance penalty introduced when using the ws2007HttpBinding.

basicHttpBinding vs. ws2007HttpBinding: Burst Strategy

The bust strategy simulates stress peaks in the number of concurrent clients. This test is typically used to simulate stress/recovery conditions for our target services. The following results were obtained when we ran a burst load profile against our sample service using the basicHttpBinding.

When switching to the ws2007HttpBinding the burst test produced the following results.

What did I learn?

I hope the results before clearly highlights the performance differences between the basicHttpBinding and the ws2007HttpBinding. More importantly, I hope this gives you an idea of all the crazy performance test you can do with the SO-Aware Test Workbench.