Nostalgia Unleashed
I was cleaning off an old hard drive the other
day and I came across the Web site of a software company I had started in
college. At the time I had a few different business ideas, it it’s funny to think
about how the ideas have changed. One of the ideas was to do standard software consulting.
This was typical “will code for food”-type stuff, where I quickly learned to always
have a contract, which is very easy to do and binding, and very easy to
get in writing from anyone with business integrity. Be wary if a potential employer refuses to sign
a contract.
Another idea I had been working on was in building a load balancing component
for reuse in other applications. I’ll talk a bit more about that when I get a
chance in the future.
One of my favorite projects was a system where I delivered sports scores,
stock quotes, news headlines, etc, to display hardware such as LED-based
scrolling ticker displays and character wall displays. There were two main
business models behind this system. The first was where a company could lease
the hardware display and data feeds they wanted from me and we would install it
and make sure that the right information displayed at the right time. The second
option was where we would make an agreement with establishments, such as major
chains like sports bars, casual restaurants, airports, etc, to deliver the
hardware and data feeds, but keep the rights to sell advertising space both on
the side of the physical devices as well as on the displays themselves, such as
scrolling text and bitmaps across the ticker hardware. At the time, this was the
most complex system I was working on, so I figured it would be fun to take a
look at how things might have changed in the past three years.
The first issue to be solved is that different data providers send data in
different ways, so abstracting out the way data is received and processed in the
system is always a good practice. I was testing out a few data provider systems
at the time, and they generally delivered their data in one of three ways:
- Over satellite linkup. This was pretty cool, except that it was a pain to set up,
and I ended up not going in this direction because it introduced a whole set
of additional physical complexities. Basically, you need a dish pointed at the
sky, and it fed data down through a cable to a converter box, and that fed out
an RS-232 (serial) feed that you could read in from a COM port.
- Over HTTP pull.
This was where you just poll a Web server for data at a given interval (some
places only allow one request every 5 minutes) and then parse it, which could
either be as a “comma-separated-value” (CSV) file or any of a bunch of others.
CSV files were easy to parse because the rows were separated by newlines
(‘\n’) and then each column was separated by commas (‘,’). I must have written
the 10 lines of code to do this a hundred times in test and trial apps.
- Over FTP push. Here you give the data provider an FTP
account, and they push data out to your server whenever it happens. The data
was usually delivered in the form of a “SLUG” which means that there was no
data in the file, but rather the filename itself contained the encoding, such
as “0001NHLNJDNYR002004F” which would decode to mean that this was the first
message of an event (usually you get updates every few minutes with the value
represented by the first 4 digits incremented each time) from the NHL game
where the New Jersey Devils were defeated by the New York Rangers by a score
of 4-2.
I ended up opting for all FTP systems because they were very easy to handle
and required little maintenance or overhead effort. I put a pretty simple system
in place where I had an app that ran in the background on the machine and would
check each upload directory every sixty seconds for new files. If new data had
arrived, it would read the data and put each set into the appropriate CSV file,
so whoever was consuming the app would only have to deal with the CSV file. The
CSV files each had different formats, depending on the type of data and the
subsets within. For example, an NHL line might look like “NHL, NJD, NYR, 2, 4,
F” whereas a golf line might look like “PGA, Woods, -1, 72, Jones, 0, 72,
Norman, 2, 71”. I knew I would eventually migrate to a database where each sport
would have a different table, but for the meantime this was very easy to work
with and useful to test before I had any live feeds.
Client requests would come in through the Web server, which seemed like the
easiest way to handle them. There were two services provided:
- The data service. This was an interface designed for client machines to
connect up and make requests through the query string of the HTTP request
(including user credentials), and a CSV file would be returned with the
requested data. It wasn’t designed to have a human interface, so CSV was good
enough and made it easy to go cross-platform. Ironically, the HTTP effort put
in on both sides was a good portion of the overall development time, whereas a
built-in SOAP Web services interface would have made my life incredibly
easy.
- The administration service. This was a simple Web UI
designed for system administrators to be able to define the playlists that
would run on each display at each location. A “playlist” was the set and order
of data to be displayed at each client site. For example, some sites might
subscribe to financial data, so they would have a playlist that would be like
“1239, NASDAQTOPGAINERS, NASDAQTOPLOSERS, NASDAQTOPVOLUME” whereas a sports
bar might have “3437, MLBNL, SPORTSAD1, MLBAL, SPORTSAD2, NHL, SPORTSAD3, NBA,
LOCALAD3437” that has sports info intermixed with paid advertisements. It also
provided a way to add and remove users, as well as define which advertisements
would be played at which venues. For example, it would be much more effective
to play beer ads at a sports bar and business ads in an airport, so this was a
key part of the service.
The biggest pain with developing these services was that I was using C++ with
the standard CGI interfaces available. For those who haven’t had the joy of
working in this environment, you would write a C++ app that would pull in the
query string from an environment variable (the oft-forgotten 3rd variable in
your main function) and then parse it, de-URL-encode it, and then do what you
need to do in the app itself, being sure to write out the HTTP response header
and the data (and you damn well better remember to put a blank line between the
header and the body or else you will be in a world of pain).
There were two intended clients for this system: the administration client
and the driver client. The administrator client used a Web browser interface, so
that was pretty easy to write off as server work (from the administrator service
above). I would have liked to have a rich client version instead, but it wasn’t
a priority item since most of the administration could be done by hand in
editing the user configuration file, which held all of the playlists in CSV
format. There was also a directory where ads were kept, but this was pretty
simple to update through FTP.
The actual driver client itself was where the majority of work needed to be
done. It was an MFC app written in C++ that would poll the server with its
credentials (every location had different credentials in order to easily
customize the playlist) and then pull down the customized playlist, which it
would parse and pass off to some core logic that would decide how to display
data. For example, an NHL score might be displayed on a scrolling ticker such
that “NHL” was in amber (yellow) on the top row, then the team and score would
be lined up as a box score with the winning team in green and lowing team in
red, and finally the game status in amber on the bottom row. Once the display
format was built, it would be passed off to the hardware driver that knew how to
transmit the proper protocol for the given hardware display, usually over serial
cable, although sometimes over TCP/IP.
Here’s an example of how the whole system looked:
Today, I would do a few things a little
differently:
- I would use C# and the .NET Framework for everything. The fact that I can write everything
in C# using the same base class libraries and APIs everywhere would make my
life incredibly easy. C# is also a much easier and more productive language to
develop with than C++ (for just about everything I was doing) which is largely
due to the .NET Framework.
- I would use Windows Server 2003 as my backend. Instead of using FreeBSD APIs to build the
file watching daemon, I would build a Windows Service that takes advantage of
the “FileSystemWatcher” object that raises events when something was uploaded.
I would also use WMI or performance counters to debug and monitor the
application, as well as event logging to keep track of anything else (or just
make sure all was well).
- I would use ASP.NET for building the Web interface. Instead of the CGI APIs to write the Web
server services, I would build a set of ASP.NET Web services that would
directly expose Playlist objects and administrator functionality. Although I
would initially wrap the administrator Web services with an ASP.NET Web app
until all of the other priorities were met, it would eventually give way to a
smart client version that consumed the same services.
- I would use SQL Server 2000 instead of the CSV files. Duh.
- I would take advantage of Windows Forms for the client
apps. I would build the administrator
app as a Windows Forms app that would use backend Web services to persist any
changes. Both apps would be delivered via no-touch deployment, making it
really easy to keep everything up to date without having to worry about
individual updates to individual machines. The integration would be done
entirely through Web services, which is pretty much automatically handled by
Visual Studio and the .NET Framework without any coding. All of this means
that I would have to write 0—yes ZERO lines of deployment-handling code and
ZERO lines of integration-handling code. That would probably cut out 60% of
the overall source investment. Also, since SOAP provides loosely-coupled
integration, I wouldn’t have to worry about the versioning of components if I
changed or added anything on the server as long as the same endpoints existed
across versions.
- I would consider making a special Windows XP Embedded client to
replace the driver PC. This could go either way, but it might just be
easier to build a lightweight hardware attachment running XPe with the .NET
Framework, serial and network drivers, and have the apps included, which would
plug into the side of the display hardware. This would result in a very
specialized system (which might not even require UI) and be much easier to
conceal in the deployment environment.
My new system would probably look something like this:
I really miss the world of hands-on software development. Thinking about all
of my old projects has made me very nostalgic, and I must admit that I’m a
little jealous of everyone who gets to actually write code these
days.