EOAST - Evolution of a software thingy - Part 2

Published Monday, August 11, 2008 4:14 PM

In a part 1 of this series, I talked about what was the purpose of coming up with this "software thingy". Now I will tackle some of the design aspects that have evolved as I have been working on it.

Just to mention the requirements again though, it must be able to gather and store information from literally any source, and not tied to one collection mechanism. It must also be able to display any data in any format, typically something that the user/administrator configures to their liking. It must be easy to develop the collection and display parts. The main engine will take care of scheduling collections and storage of the data. Data can be pushed to it, not just collected. Must be reliable and "non blocking' in the event of a bad collection or display module.

It needs to allow me to exercise some of the new fandangled technologies but not at the expense of functionality.

Whew. Thats it in a nutshell. So what have I come up with so far (remember, being a pet project, its really a moving target). The diagram below shows a conceptual diagram of what needs to occur and the logical boundaries of components.

image

Obviously, a common element to house each item of information would be a good way to go initially. Something that caters for all types of information, with indicators pertaining to its source. As long as the data can be stored, and ideally searched, the display of that item is almost irrelevant for now.

Here is a sketching of the information items I initially wanted to capture/cater for.

image

So with the above things in mind, it seemed appropriate that I dub this piece of software the "Information Aggregator". Yes, like an RSS aggregator but with general information item in mind, which may or not be RSS items.

Continuing with the high level design though, extrapolating the above 2 diagrams further, and taking into account how I want the entire process to operate, below is an initial diagram of some components/services and how I expect interfaces and information flow to occur.

image

You can see that the "Engine" is the core component. The idea is that it maintains a list of all collection agents and display agents. The engine will co-ordinate scheduling of collections from the various agents, and also allow registered agents to push information to the engine.

Separately, upon receiving information or based on request from display agents, the engine will push information to any display agents, that have been associated with a particular collection type.

You can also see the interfaces that I have begun to design here. I will delve more into this in the next post, however the main interfaces being IPluginModule, IInformationPush and IInformationPull. IPluginModule being the interface that all compliant modules must implement. IInformationPush and IInformationPull are optional (although at least 1 makes sense) depending on the type of the module.

Note: I am aware you could probably use one of the composite application blocks to achieve an outcome similar to what I describe here, but I didn't want to rely on anything like that. Mainly because this is about learning and playing with stuff, as much as getting to an end solution.

So the eventual concept being built, is that information gets collected from some collection providers, lets say a provider called GlavsEmail and GlavsNoteEntry.

I have 3 registered display providers:

  1. A balloon notification
  2. A full detail display
  3. A summary display.

I map the GlavsEmail information provider to the balloon notification and the summary display. I map the GlavsNoteEntry to the full detail display.

I set up the schedule for the GlavsEmail collection to occur every 1 hour. I set the schedule for the GlavsNoteEntry o be manually initiated (ie. by the information provider itself).

The engine should now perform the necessary collections when required, and when elements of GlavsEmail are gathered, display those using the 2 mapped display providers (balloon and summary), and the GlavsNoteEntry gets displayed using the full detail display.

At any time I can change this mapping. Also, all of the information from those sources is stored in SQL for easy searching and extraction if required. All I would typically have to do is register the collection and display providers, map them accordingly, and set up the access credentials for the collection provider (email in this case).

Thats it for this post. Next post I will delve more into the design of the interfaces, and technical details of the software (might be a few posts actually). Mind you, its still a moving target and not complete (like most of my personal pet projects) so don't expect enterprise level detail and a complete publicly consumable API.

The post after that will probably show you the product in action :-)

Comments

# Chris Parnin said on Monday, August 11, 2008 10:48 AM

Yes, the information singularity.  I think everyone had this sort of architecture in mind -- you're actually trying to code it.

Some things to consider:

1) Many aspects of this have been tried before in the "databinding" sense.  You should compare your architecture to see what separates you and what you can borrow from that slant.

2) I think the Data Storage can need expansion.  What if my some of my data is on a flickr account?

3) Non-local capture and displays:  What if I want to send my information to my refrigerator, or capture diagrams from my digital pen.  Can I get my vacation picks (on my sick day) to display on the photoframe in my living room but not my office?

This is in some sense a "disconnected" database problem where things have to be synced up when connections become available.  In many cases this is due to temporary limitations in capture technology, but still a long-term problem.

# Chris Parnin said on Monday, August 11, 2008 12:46 PM

Yes, the information singularity.  I think everyone had this sort of architecture in mind -- you're actually trying to code it.

Some things to consider:

1) Many aspects of this have been tried before in the "databinding" sense.  You should compare your architecture to see what separates you and what you can borrow from that slant.

2) I think the Data Storage can need expansion.  What if my some of my data is on a flickr account?

3) Non-local capture and displays:  What if I want to send my information to my refrigerator, or capture diagrams from my digital pen.  Can I get my vacation picks (on my sick day) to display on the photoframe in my living room but not my office?

This is in some sense a "disconnected" database problem where things have to be synced up when connections become available.  In many cases this is due to temporary limitations in capture technology, but still a long-term problem.

Leave a Comment

(required) 
(required) 
(optional)
(required) 

This Blog

Syndication