Becoming asynchronous – The first step towards distributed applications

In my previous blog post I argued WCF was not the most usable and most easy to learn way for communication in distributed applications. This is due to its focus on synchronous communication (even though you can do asynchronous communication as well).

Distributed applications by their very nature cannot communicate synchronously. Their parts are running on different threads (or even machines). And communication between threads is asynchronous. Always. And if it appears otherwise then there is some machinery behind the scenes working hard on that.

Now, if distributed functional units are running on different threads, why would anyone even try to hide that fact? Why do Web services, Enterprise Services, .NET Remoting, and also WCF abstract away this inherent parallelism? Well, it´s because asynchronous communication and asynchronous systems are harder to reason about.

Great, you could say. That´s what I want: easy to reason about systems. Isn´t that purpose of abstraction? Making complex things easy?

Well, sure. We love C# as an abstraction for how to command a processor. Nobody would want to go back to low level assembler coding.

But not just any abstraction is a good abstraction. Some abstractions are, well, “unhealthy” or plain wrong. Think of the abstraction “man is like a machine” and all its implications for the health industry.

Abstractions usually have a limited scope. Treating humans like a machine when “repairing” a wounded finger might be ok. But when treating cancer, physicians surely should view their patients differently.

So the question is: When to stop using an abstraction? Or whether to use it at all.

That´s what I´m asking myself regarding WCF. When should I stop using it and start coping with the asynchronous nature of distributed systems myself?

My current answer is: I should not even start to do remote communication with any synchronous API. And my reason is simple:

1. The communication paradigm (sync vs async) is very fundamental to any application. Switching it in mid course is usually very hard.
2. Async communication is the sine qua non for truely scalable systems.
3. Our industry is notoriously bad at forseeing the course of development of applications. Many start small, as a prototype – just to become mission critical with the need to scale far beyond the initial purpose. (I´m sure you too know at least one of those MS Access applications that have become unmaintainable but need to live on because today maybe hundreds of people are depending on them.)

Taken together this means: Systems start small, even when distributed. So they use a sync communication API. Then they grow, need to scale, and thus need to switch to async communication – which then is probably infeasible or very hard to accomplish.

So when I´m saying, “You should not even start to do remote communication with any synchronous API.”, I´m just trying to apply lessons learned. If we know switching from sync to async is hard, and if it´s very likely this switch will be necessary some time in the future (because, well, “you never know” ;-)… then, I´d say, it´s just prudent to not go down the path, that might be easier at the moment, but will be much harder in the future.

There´s a time and place to do synchronous communication between functional units. And there´s a time and place to do asynchronous communication. The latter being always the case when developing multi-processor code or distributed code.

The earlier a developer learns to see this boundary the better.

Enough with theory for now. Let´s get our hands dirty with some code.

Switching from sync to async using the CCR

How can you start with asynchronous programming? Let me say that much: Don´t even try to manage threads yourself! Instead leave that to some framework or runtime. Concentrate on an easy programming model.

The programming model I´m going to use is purely message oriented and based on “pipes” through which those messages flow. A client “poures” message into such a “pipe”, and a service at the other end “ladles” them out in order to work on them.

You can implement such a programming model using queues. But I recommend you don´t think about doing it yourself ;-) Instead use an existing component like Microsoft´s Concurrency Coordination Runtime (CCR). It´s part of Robotics Studio, but can also be use standalone. All I´m gonna show you in terms of async and distributed communication will be based on the CCR. I just love it :-)

Here´s the notorious Hello World code snippet to show you, how messages are sent using the CCR:

…

The “pipes” I was referring to are called Ports in CCR lingo. They are queues – but thread-safe ones.

So this is how a client sends a message off to some service. It just stuffs it into a Port and forgets about it. The message is processed asynchronously by the service.

To let a service listen for messages on a Port is somewhat cumbersome using just the CCR. But I´ll show you anyway, even though later on abstraction layers on top of the CCR will shield you from such details (most of the time).

Assume there is a service method like this:

Never mind its simplicity. For the purpose of explaining the communication basics it´s sufficient. What you want to learn is how to make it listen on a Port. Here´s how:

You need to set up a so called Receiver on the Port. The Receiver is associating the service method (ProcessText()) with the Port (p) as an event handler. The DispatcherQueue then is used to schedule event handling by the service method on some thread from a pool. Messages posted to the Port are automatically processed asynchronously and in parallel on background threads.

But as I said: Don´t feel frustrated right now. You won´t need to wire-up event handlers like that very often just to do distributed communication the async way. If you nevertheless want to learn more about the CCR, feel free to browse its documentation.

The only thing you have to remember is: Async communication flows through CCR Ports from clients to services as messages. You can even call those Ports channels, if you like – and feel at home since WCF has channels, too ;-) for the same reason as the CCR. Because WCF is message oriented and deep down internally even works asynchronously. It just does not show that to you very often or in an elegant way.

Why asynchronicity for distribution?

Before I move on to API details a quick word about why I think asynchronicity is fundamental for communication in distributed applications. Look at this code:

What expectations to do have about how the service is delivered? Think about it for a moment…

If you´re like me, you probably thought at least…

• ProcessText() better be fast because the client is waiting for the result.
• You need not bother to think about how often to call the service object. Why should you?
• There is no doubt there is an object where s is pointing to. So there is no doubt the service ProcessText() will be delivered.
• The order the client is giving the service will of course reach it, and the result the service produces will sure reach the client.
• There is of course no security risk in calling the service. Why should there be?

Or maybe only the first thought occurred to you? That would be quite naturally, I guess. Because why should you even think about the other points at all? They are non-issues for method calls.

And that´s what I find important: Since the usual Web service-, .NET Remoting-, Enterprise Services-, WCF-call is a method call like any other local method call, developers tend to don´t even think about all that what is so different in remote communication.

• You cannot assume any particular performance from a remote service because you don´t know anything about its load at call time. So your client better is prepared to wait just a little longer for a response.
• Since remote communication is 3 or more orders of magnitude slower than local stack based communication you need to very cautious about how often to call a service. Roundtrips are very expensive.
• You cannot be sure a remote service is running when you issue a call to it. Maybe it has crashed, maybe it hasn´t started yet, maybe it has been taken offline for whatever reason… So waiting for a quick response is not such a good idea.
• Even if the remote service is running you don´t know if the client can reach it. Maybe the connection is down? Maybe the connection is going down after the order is issued but before a result could be returned? Another reason why you should not rely on a response to be delivered in a time frame it would be worth while waiting for on the client side.
• And what about eavesdroppers on the line between client and service? Have measures be taken to thwart them?

As long as you cannot decide by looking at a service usage whether it´s a local service or a remote service you can´t be sure if all this has been taken into account. That I find very disturbing. To me a remote service usage through a regular method call like above thus is kind of misleading, fostering falling prey to fallacies, or simply lying. It´s lying about very basic properties, because 99% of the time we´re looking at method calls and don´t have to bother about all this.

Now, that´s all different when you see this in a code base:

You immediately realize: This is not just any ordinary request for a service. This is special, because it´s asynchronous and clearly message oriented. And then you start thinking about whether the properties of this service usage are really like what you expect.

1. Performance is not so much of an issue anymore because the client is not waiting for a result.
2. True message orientation is a tad more cumbersome to use. This makes you (more) aware of the price you´re paying when requesting remote services. You automatically become cautious about roundtrips. This will guide you when designing a service contract.
3. If the remote service is not running, the request can easily be put into a queue which can be volatile or persistent.
4. If the connection to the service is down or goes down, then queues can help again. They can even be transactional.
5. Since you´re aware of the nature of the connection to the service you are more prone to give security at least a thought.

You see, I don´t want to take away the usual sync method call for remote communication from you lightly. I think there are at least 5 good reasons why distributed communication should always be async. Always.

But of course, you don´t want to pay too high a price for such “honest communication”. So let´s see what we can do about usability when going asynchronous. Stay tuned for my next blog post…