brady gaster

MetaTechture - an idea's origin

I've been thinking for some time about architectural ideas. I was all about Session's concept of the SOA, the Fiefdom/Emissary model put forth by one of the fellow webloggers, and all of the MS Patterns and Practices documents that've been put forth.

Some time ago I wrote a post about using the Provider model for workflow, and my idea-basket began to fill up relatively rapidly. Since that posting, I've been working a lot, drawing pictures, and conceptualizing an architecture that, if implemented properly, would accomodate a few goals I've had in my brain for the past few years.

The Origin:
It all began with a particular client who told me that their application had one single business rule:

There are no business rules. Our users will be making them up as they go along.

You can imagine the blank stares that soon stretched across my face and the face of my colleague, who were the sole proprieters of this doomed idea. From the very outset of the project, I knew I'd have to come up with something just shy of impossible - a system that would allow for change at any point in its lifecycle, according to the needs of the users, on a moment's notice.

I still hold true that such an architecture is next to impossible, but with a little thought, a lot of creativity, and a whole bunch of pictures that form the mess strewn about my apartment, I think may have come somewhat close to the beginnings of such an architecture, and I've chosen this forum as a place to unleash it onto the world, request feedback, evoke some heated debate, and hopefully, make it pretty awesome.

What do I Intend to Solve?
I think that every good idea begins as frustration. Meaning, you find yourself frustrated by something over and over again. You think of ideas to solve the problem, and run aground time and time again as you sail around looking for answers. It is in this frustrated blur that the answer escapes you, and only later on, when you're clear-minded, can you look back on the point of frustration and find a solution that makes sense. This architectural idea - what I'll refer to from here on out as the MetaTechture (an architecture “about” or “for” other architectures, plus it seemed pretty catchy when I first said it) - began as a by-product of these frustrations. In fact, the core components of MT arose one by one as a result of my repeated frustrations with certain aspects of enterprise development. Given the fact that these frustrations were the major contributing factor in my decision to spend facets of my life for about 3 years drawing, reading, researching, and thinking about how I could accomodate these prolific problems, I'll begin by presenting these major complaints for you, and address each independently in terms of how “it would be nice” to be able to forever solve the problem in and of itself.

Problem #1: Requirements Change (or just plan suck right off the bat)
We all know the pain of changing requirements. Alan Cooper claims it to be the major hurdle in any and all application development paradigms. Each and every colleague with whom I've had a beer begins to tell their own version of the same horror story at some point during the conversation. It is, sadly, a fact of life for those of us who make this industry our home. Sometimes you get them up front and they make sense. Whether a result of process alteration (during your development time on any given project), poor communication (even if you do use XP properly), or the old favorite “now that we see it, we have additional ideas we'd like to see implemented” (wink wink at the client), this factor is the major contributor to MT. To accomodate this problem into MT, I'll devise a way of controlling flow - workflow, if you will - throughout a system; in fact, I'll objectify the very concept of a process step within the MT framework and introduce the notion of changing processes on the fly.

Problem #2: Code has to be Extended Later
Most clients realize this potential problem. They hire us, we write the application, it exceeds their expectations (and requirements, since they probably fell prey to Problem #1), and we move on. Then, the client realizes that they have virtually no idea, despite how awesome our documentation may be, on how to extend or maintain the application. You've probably been on at least one end of this conversation:

Customer: “What happens when our business processes change? Can the application deal with that?“
Developer: [Stammers] “Well, that depends...“
Customer: “On?“
Developer: [More Stammer, then provide the formula response] “On how complex the process change is. If we've hardcoded logic in the places where those changes are emulated within the system, the code will have to be altered.“
Customer: “Is that included in your maintenance agreement?“

Problem #3: Too Many Database Platforms, Too Little Time
Sure, this one isn't so big as a result of .NET's limitless support for virtually any database driver/structure/idea. With support for everything all the way back to the ODBC world, we've got a nice little series of packages we can use to access databases. This problem, of course, gives rise to the concept of a Generic Data Access Layer. MT has a component which allows for this paradigm, and as a result of being the simplest-to-understand, will be the first component covered in this series. I'll level .NET's use of OO, the old concepts of Interface Implementation, and tie it all together with a nice dosage of the Provider model to make it easy for your applications to talk to any database - or a farm of them if needed.

With these problems out of the way, I'll posit perhaps the most important aspect of MT:

When you think like a computer, you fail to address the problem resident in most business logic: persistent alteration and change. You can't set a program into motion with a few-hundred-or-so rules and expect that, when those rules fail to validate as you expected, the program will continue to execute. It isn't very real-world of you, now is it? The world isn't written in ones and zeroes, and your application logic should expect and deal with flies in the ointment.

MT will attempt to solve these stated problems. Hopefully the readers of this series will put forth some experiences of their own to add to the problem list, thereby only making MT a more founded, solid architectural idea. From here, the posts in this series will traverse down the ladder of architectural complexity.

First off, I'll try to appease Problem #3. In the days to come, the second article in the MetaTechture series will put forth a relatively simplistic data access architecture, somewhat based on the SqlHelper class presented by the Data Access Building Block. From that point, I'll discuss the notion presented originally in this post, which focuses on dynamic workflow control via the Provider model. I've expanded this idea almost to the point of being application-server-ish, and will attempt to explain the idea pretty clearly through a few individual posts, which explain each aspect of the workflow model in MT. Finally, I'll go into some length explaining the Plugin framework within MT, and discuss the aspects of flexible plugin loading, assembly inspection, and so forth. Finally, I'll exemplify all of these aspects in a single, simple case study which will facilitate each aspect of MT to exemplify its usefullness.

And yes, I invite any flames, suggestions, bashing, or “so-and-so product already does that.” Yet, my hope is that I can inspire you through creative use of OO concepts, a little elbow grease, and some old-fashioned head-down coding.