Software Cells #9: Applications as Holons - Becoming a little more systematic

Software Cells started out in the concrete and tried to make small to medium scale software architecture more tangible. Software Cells thus were a bottom-up approach.

But as it turns out, now I also find them usefull on a more conceptual and general level. The have become a top-down model for describing/building software and have the tendency to become more systematic on several levels. I realized this when I read about Arthur Koestler´s Holons the other day (see here for an introduction: http://www.worldtrans.org/essay/holarchies.html). The picture of an Holarchy - a system of several levels of nested Holons - immediately reminded me of my picture of the Software Universe (see picture at end of posting).

In Koestler´s terminology Software Cells, Solutions, but also Assemblies are Holons. Each kind of Holon on a certain level consists of Holons of the next lower level and at the same time is part of a Holon on the next higher level. The Software Cells model thus defines its own Holarchy with 7 levels from the smallest Holons to the largest ones:

1. Method
2. Type
3. Assembly
4. Component
5. Application
6. Solution
7. Society

(As you might have noticed, I neglected the level of statements which was part of the Software Universe. I think that´s ok, since statements themselves are predefined by a programming language. Their "shape" and interface is not under our control. But from Methods on upwards the Holarchy we are in control. Methods are the lowest level Holons we can "compose".)

To me, rooting Software Cells in the Holarchy concept is valuable since it brings more systematic thinking to them. And more systematic thinking is always good. It´s a step back from the concrete and brings into view the context. It allows you to reason about things you know less about by carrying forward knowledge. It allows you to find patterns and see developments which are broader than a particular "thing" you usually focus on. A systematic approach also let´s you find categories of things which raise the level of abstraction. Reasoning about few categories is easier than reasoning about a million different things falling into those categories.

Let me try to apply a little bit more systematic thinking to the Software Cells Holarchy. My first point of view is structure of Holons.

I´d say, each Holon (from Method to Society) can be viewed as a 2-tupel (dupel) consisting of (interface, implementation). That´s very easy to see if you look at the lowest levels of the Holarchy, methods and types. Methods have a signature as an interface and a body as an implementation. Types have public methods and fields as their interface and the method bodys as their implementation. An Assembly´s interface are the public types, and all types with their implementation make up the implementation.

With this generalization in place we suddenly see a difference in how concrete interfaces are on the different levels. Implementations are always easy to identify, but the interfaces as the (syntactic) contracts for interacting with the implementation get less tangible the higher you get in the Holarchy. A Method clearly ties the interface to the implementation. Even an Assembly today ties together interface and implemenation. In fact it was an important goal for Microsoft to get rid of the separation of metadata (interface) from implementation. But what about Components, Applications, and Solutions? Component metadata is distributed across several Assemblies. The same is true for Applications, and its worse for Solutions. There is no "one location" to find the metadata for a Component, or Application, or Solution.

And I think, that´s bad for the same reason it would be bad to separate a Method´s signature from its body. Systematic thinking thus leads us to the proposal to define a means for assembling the interface of a Holon on the levels from Component up to Society. For a Component this seems to be pretty straightforward: just put all public CTS interfaces (which are implemented by the Types distributed across the Component´s Assemblies) into a single Assembly. But what to do for an Application or Solution? I don´t know. Maybe a single large WSDL/XML Schema document would do? (Which could be retrieved from a single URI.)

This brings me to another view on the Holarchy: Control Flow between Holons. Holons can be split into two categories:

• Holons communicating via the stack.
• Holons communicating via streams.

This view is tightly coupled with another view: Holons can also be split into two categories regarding their "tangibility". There are physical Holons and virtual Holons. Physical Holons are those, that have clear physical "boundaries", Method, Type, Assembly, Component, Application. I can point at them and say "These lines of code make up the abc Method." or "Ths file is the def Assembly Holon." or "This process is the xyz Application Holon."

But where are the physical boundaries of a Solution or Society? I find it hard to to point at "something" and say "This is the Solution Holon."

The lynchpin of physical vs virtual Holons is the process and thus the Application. From Applications upward on Holons are virtual - and communication switches from stack to stream.

But what does it mean, when I say, "communication is stack/stream based"? It means, a stack/stream is the medium used to communicate between Holons on the same (!) level of the Holarchy:

With this picture we now can understand maybe a little better, why there is so much hype around technologies like Web services or Indigo. They are so to speak "growing pains" or "symptoms" of moving up software development in the Holarchy. Most software developers have lived until now in a world of single Application software. They thus needed to be concerned only with the stack based RPC style communication. And when they wet their feed in multi Application software (or simply Solutions), Microsoft tried to make it easy for them with DCOM and MTS/COM+. But nevertheless they moved from the old stack world to the new stream world where messages needed to be exchanged for communication.

However, streaming communication (across memory spaces) is very different from stack communication (within a single memory space). And not all the differences could be glossed overy by DCOM et al. Neither can .NET Remoting gloss over all differences or Web services (in their original form back in 1990). So what the Software Cell Holarchy allows us to see is the break or transition in the levels of hierarchy with respect to communication. (Please understand me correctly, I´m not claiming I´ve discovered something new here. I just claim, that the Software Cell Holarchy helps a great deal to make visible what exists and reason about it.)

Finally, let me assume another viewpoint: The history of software development can be seen as a continuous rise in abstraction. An abstract type is more abstract than a set of methods. A component is more abstract than a set of types. etc. Fundamental for the ever higher levels of abstraction are the interfaces of the Holons on each level. The interfaces hides the details of implementation. Now, since succinctly describing a service using an interfaces is so important for abstraction, and Holons are duples containing an interface and at the same time are "containers" for lower level Holons, the history of software development is also an evolution of aggreation and containers.

A Type is a container for Methods and OOP brought them about. So OOP is at the beginning of the Software Cells Holarchy. Then came DLLs as containers for Types. Strangely, Components lack a container, but Applications (or Hosts) are containers for Components. (Maybe .msi files could be at least the deployment containers for Components?)

With each new container, we were able to assemble our software using more coarse grained parts and on a higher level of abstraction. The end of this development, though, so far seem to be Components. 3rd party software like a WinForms data grid or O/R Mapping engine are Components. Since the old Visual Basic VBX components a flourishing market has come into existence, where prefabricated Components are sold. Building true Components within a project, though, is not yet equally popular.

But what will be the next step in this evolution? Clearly it is to manufacture whole Applications and then Solutions and sell them as prefab parts for new software. However, what makes this more difficult than stepping up from Type to Assembly, is, that Applications are on the stream based communication level of the Holarchy. That means, the whole (or Holon) assembled from different Applications is not contained within one process (memory space) and not even necessarily running within the same network or on the same platform. The assumptions a prefab Application can make about the whole are much less than a type can make about an Assembly. A Host as the ultimate container for Holons from Component down to Method is a very stable and straightforward environment. Heterogeneous networks, though, are a completely different environment. There are simple so many more "moving parts" and parameters to account for.

And that leads me to the prediction: Where true Component based programming has not taken off in most software shops until today, the next level of software modularization, where whole Applications are the components, will take at least the same time to materialize. For me that means: Despite all the SOA hype we´re another 5 years away from a broad and natural acceptance of Software Solutions. It took from ca. 1992 to 2005 to get developers ready for "ordinary" Components. The technologies are in place, IoC/Microkernel infrastructures allow you to stitch together your Applications from Components/Assemblies very easily. Eclipse is maybe the best example for how successful this approach ultimately can be.

If you see 2000 and Web services as the starting point for the evolution onto the next level and it will take developers almost the same time to solidly understand and broadly use the new concept of Applications as components, then not before 2010 we´ll see widespread acceptance.

But sure, this is speculation. We´ll see what happens.

The trend towards more and larger prefab parts is unbroken in the non-IT world as can be seen here: http://www.modularesbauen.com/berichte/Das_Haus_vom_%20Band.pdf. This document describes a concept for the family home of the future. An affordable house (ca. 50.000 EUR) composed of prefab modules. Likewise, I guess, we´ll see more coarse grained prefab parts in the IT- and software world. And the Software Cells Holarchy is showing us, on which levels that´s to happen and how they relate to existing concepts. Thus I think moving Software Cells little more in the direction of a systematic approach is valuable.