August 2008 - Posts

EOAST - Evolution of a software thingy - Part 2
Monday, August 11, 2008 4:14 PM

In a part 1 of this series, I talked about what was the purpose of coming up with this "software thingy". Now I will tackle some of the design aspects that have evolved as I have been working on it.

Just to mention the requirements again though, it must be able to gather and store information from literally any source, and not tied to one collection mechanism. It must also be able to display any data in any format, typically something that the user/administrator configures to their liking. It must be easy to develop the collection and display parts. The main engine will take care of scheduling collections and storage of the data. Data can be pushed to it, not just collected. Must be reliable and "non blocking' in the event of a bad collection or display module.

It needs to allow me to exercise some of the new fandangled technologies but not at the expense of functionality.

Whew. Thats it in a nutshell. So what have I come up with so far (remember, being a pet project, its really a moving target). The diagram below shows a conceptual diagram of what needs to occur and the logical boundaries of components.

image

Obviously, a common element to house each item of information would be a good way to go initially. Something that caters for all types of information, with indicators pertaining to its source. As long as the data can be stored, and ideally searched, the display of that item is almost irrelevant for now.

Here is a sketching of the information items I initially wanted to capture/cater for.

image

So with the above things in mind, it seemed appropriate that I dub this piece of software the "Information Aggregator". Yes, like an RSS aggregator but with general information item in mind, which may or not be RSS items.

Continuing with the high level design though, extrapolating the above 2 diagrams further, and taking into account how I want the entire process to operate, below is an initial diagram of some components/services and how I expect interfaces and information flow to occur.

image

You can see that the "Engine" is the core component. The idea is that it maintains a list of all collection agents and display agents. The engine will co-ordinate scheduling of collections from the various agents, and also allow registered agents to push information to the engine.

Separately, upon receiving information or based on request from display agents, the engine will push information to any display agents, that have been associated with a particular collection type.

You can also see the interfaces that I have begun to design here. I will delve more into this in the next post, however the main interfaces being IPluginModule, IInformationPush and IInformationPull. IPluginModule being the interface that all compliant modules must implement. IInformationPush and IInformationPull are optional (although at least 1 makes sense) depending on the type of the module.

Note: I am aware you could probably use one of the composite application blocks to achieve an outcome similar to what I describe here, but I didn't want to rely on anything like that. Mainly because this is about learning and playing with stuff, as much as getting to an end solution.

So the eventual concept being built, is that information gets collected from some collection providers, lets say a provider called GlavsEmail and GlavsNoteEntry.

I have 3 registered display providers:

  1. A balloon notification
  2. A full detail display
  3. A summary display.

I map the GlavsEmail information provider to the balloon notification and the summary display. I map the GlavsNoteEntry to the full detail display.

I set up the schedule for the GlavsEmail collection to occur every 1 hour. I set the schedule for the GlavsNoteEntry o be manually initiated (ie. by the information provider itself).

The engine should now perform the necessary collections when required, and when elements of GlavsEmail are gathered, display those using the 2 mapped display providers (balloon and summary), and the GlavsNoteEntry gets displayed using the full detail display.

At any time I can change this mapping. Also, all of the information from those sources is stored in SQL for easy searching and extraction if required. All I would typically have to do is register the collection and display providers, map them accordingly, and set up the access credentials for the collection provider (email in this case).

Thats it for this post. Next post I will delve more into the design of the interfaces, and technical details of the software (might be a few posts actually). Mind you, its still a moving target and not complete (like most of my personal pet projects) so don't expect enterprise level detail and a complete publicly consumable API.

The post after that will probably show you the product in action :-)

EOAST - Evolution of a software thingy - Part 1
Monday, August 4, 2008 4:48 PM

Its been a little while since I have blogged as I haven't had very many value items to add to the blogosphere lately. However, I have been steadily (albeit slowly) working away on a little software project that may or may not see the light of day, but it has been a fun learning experience, and may eventually come to something. If nothing else, its a good testbed for a number of latest technologies.

I decided to chronicle some aspects of this software "thingy", hence the blog title "Evolution of a software thingy" - EOAST (you gotta have an acronym otherwise you don't exist :-) )

So this first post is the requirements if you will, the business drivers behind what and why I decided to do what I did.

Context

If you have read any of my previous rants, you'll note that I am not the biggest fan of the latest incarnation of outlook. I use it daily, and rely on it, but its slow, often locks up (particularly on slower links) and generally sucks the life from my system.

Additionally, like most people, I use a number of various tools to collect information for either storage, perusal, reference and a bunch of other things.

The collection of various information pieces shows a lot of similarity with each other, the main difference being collection frequency or scheduling, management of the information, and its presentation. For all the tools I use, none of them are exactly how I want them. Each one has different parts I like and dislike and merging the information they gather into a manageable unit is tedious, if not almost impossible in some cases. Think blog aggregators, email, twitter etc. Essentially, they all collect some information at different frequencies or schedules and display it for manipulation or searching, sometimes storing it for later retrieval or whatever.

That's the pain point part. I also wanted to have a purpose for some of my technological investigations. We all know that you can play with a new technology like Entity Framework, or Linq to SQL and make it do what the examples do. However, when hard pressed to do something that meets a requirement, not a technology demonstration, and suddenly the game changes, and efficiencies and/or deficiencies are far more apparent. So essentially I wanted to play with some new tech as well to get a real feel for its capabilities.

Requirements

So with that in mind, my requirements for my "Software Thingy" are as follows:

  1. Must be able to collect information at various scheduling intervals, whether monthly, daily, hourly, every minute, or manually initiated. Additionally, it must also be able to push information, rather than simply request via a schedule. Think Microsoft exchange (push) and blog readers (pull).
  2. it must store this information that it collects into a reliable store, SQL Server being the obvious candidate. This will cater for large volumes and easy searching. (Think problems with large Outlook PST files).
  3. It must be reliable to handle badly written modules that it interfaces with. If something is taking too long, or just plain dead, the main application/engine will not die as well and render itself unusable (ummmm....Outlook again :-) ).
  4. It must not be tied to one method of collecting information. It must be flexible enough to gather information from anywhere. Blogs, twitter, POP email, exchange email, manual entry into  some data capturing app, text files, anything.
  5. it must not be tied to one display method. I want the ability to determine how particular information is displayed. I'd like to be able to map the gathering of information from one source, to a certain display method, or even multiple display methods, all administratively.
  6. I don't want to do it all. I don't have that much time or smarts. I would like to make something very easy to develop display modules or information gathering modules so that I can get perhaps a WPF guru to make me a slick WPF display module, without having to care where the info comes from and how its collected. Similarly for the information gathering. I'll write the items that pertain to me, and allow others to easily bolt on a different form of information gathering and not necessarily care about how its stored nor displayed.
  7. I want to utilise technologies such as WCF (my favourite), Linq to SQL, WPF and any other thing that makes sense.

Next up will be the initial design that I have come up with. Stay tuned.

MSFeedsSync - feeds Synchronization has stopped working issue
Sunday, August 3, 2008 9:59 PM

Recently (and quite randomly I might add), I had an error dialog popup continuously, every 5-10 minutes which read:

"MSFeeds Synchronization has stopped working (2054)"

It was driving me crazy and was unsure how to resolve. There is actually not that much on the net about but this post got me going in the right direction.

Basically, I first disabled the service by typing at the command prompt:

msfeedssync disable

Then I deleted all my files (which included many feeds I was no longer using) from the following dir:

\Users\%username%\AppData\Local\Microsoft\Feeds

then re-enabled the feeds service by typing:

msfeedssync enable

And all was good again.

More Posts

This Blog

Syndication