SOA Design with Agile methodologies

Service Oriented Architecture (SOA) and Agile - two of the current "hot topics" in IT.  The interesting thing is that even though they're fundamentally different - one being a set of architectural principles, the other a set of methodology principles, one of the key goals and benefits of both is enabling change.

Now, there are clearly differences in the goals (enterprise integration vs. fail early, etc.), but that's beyond the scope of this entry.  What this is about is that these are two sets of principles that are both useful and appropriate in today's technology landscape.  So, a natural situation to arrive at is where you want to transform a company to use both Agile and SOA.  This raises an interesting issue, though...

  • Agile methodologies promote an incremental, iterative approach to development of functionality (including method signatures), with visibility of the impact of change given through test coverage.  Basically, working on the premise that, change is cheap if supported correctly
  • SOA promotes a well defined service interface through contracts - these contracts are aligned against business processes, not implementation details.  The rigidity of the service interface, and its non-technical alignment allows for internal change without impacting consumers of the service.  Changing a service interface is an involved process, though; rather than an outright change, a versioning mechanism is needed due to the immutability of contracts, and therefore a migration process is required.  The visibility of service usage is also very difficult to monitor as it could be external organisations rather than internally controlled systems.  The upshot of these two factors is that change to the interfaces of a service are relatively expensive

So, whilst the internal development of services and service-consumers is in-line with Agile, the creation of service interfaces isn't - an incremental, iterative approach to those would cause serious versioning and compatibility problems, especially with remote (3rd party) consumers.  Similarly, imagine you're on the receiving end of another company that has an Agile approach to a service interfaces you're consuming; the contract you're adhering to would shift under your feet continually, introducing additional cost.

Thankfully, there is not only a solution to this problem, but it is also one that actually benefits Agile methodologies.  The solution is to make use of the incremental aspects of Agile, without the iterative approach when designing service interfaces.  Basically, the service interfaces should be defined just-in-time; when the first requirement for a certain piece of functionality arises, the definitive implementation of the interface for that particular service method and no more is derived.  The removal of an iterative approach to interface design is important not just for cost reasons - one of the key factors in the success of an SOA is how accurately the service interface mirrors the real world process.  If deriving a clear definition of this interface cannot be done, then it generally means that the underlying process isn't well understood.

As for the benefits of this to Agile, it's long been my opinion that the largest problem with such approaches are the introspective facets of them - the virtual disregard for what isn't clearly defined and within the primary context.  Service interfaces act as change boundaries for these introspective "pockets", containing them from unintentional impact on other such "pockets".  This allows iterative approaches to be taken to the component- and object-oriented internals of services, without incurring the cost of repeatedly integrating between potentially external, non-visible systems.

2 Comments

  • SOA and agile are very compatible if you choose to work that way. This is the approach I would suggest:

    1. Adopt Agile Model Driven Development (AMDD) techniques, www.agilemodeling.com/essays/amdd.htm, to help derive the interface definition.

    2. Iterate during development as much as you need.

    3. Incrementally release services into production once they've stabilized.



    - Scott

  • Good feedback, Scott... cheers. :)



    In an enterprise, I think that point (2) is the one where there are hidden pitfalls. Internal, iterative development would work fine in relatively small/closed systems. One thing that I didn't really mention in the post was that I was talking about larger enterprises, specifically a bank in my case, where there are potentially several projects at once vying for control over the shape and direction of a particular service.



    Coordinating these projects would be nigh-on impossible, leading to a long-term lack of stabilisation if a self-organising (emergent) approach were taken. From experience, what occurs is method off-shoots - multiple, similar versions of the same thing, all evolving independently, breaking the SOA model, and giving you a whole world of legacy maintenance. The problem is thus making each project aware of the requirements of the other, whether this is done by having separate, non-project-centric control of the service interface (an "OK" approach, and acceptable for now, but a bit too much like command & control in the long-term, really), or introducing all business-wide requirements of the interface to the project, regardless of whether they apply (really not good).



    To me, this is the inherent "introspective" issue of Agile methodologies surfacing here. Now, if the services didn't have to be exposed externally (i.e. you had 100% visibility, test coverage, and continuous integration of your systems), there is a solution that we've come up with independently that's very close to AMDD... Architectural overviews (1 sheet of A4 diagrams) produced by each team at the end of each iteration. These are created in a common language by each team so that enterprise architects can assimilate them into a view of the whole enterprise, and the forces currently on such things as service interfaces can be seen. With "black holes" in the system - tactical solutions that happen out of sight, as occurs in any large enterprise, and external, third party consumers of services, you don't have the visibility to make such a model work. :(

Comments have been disabled for this content.