June 2003 - Posts
I was cleaning off an old hard drive the other
day and I came across the Web site of a software company I had started in
college. At the time I had a few different business ideas, it it’s funny to think
about how the ideas have changed. One of the ideas was to do standard software consulting.
This was typical “will code for food”-type stuff, where I quickly learned to always
have a contract, which is very easy to do and binding, and very easy to
get in writing from anyone with business integrity. Be wary if a potential employer refuses to sign
a contract.
Another idea I had been working on was in building a load balancing component
for reuse in other applications. I’ll talk a bit more about that when I get a
chance in the future.
One of my favorite projects was a system where I delivered sports scores,
stock quotes, news headlines, etc, to display hardware such as LED-based
scrolling ticker displays and character wall displays. There were two main
business models behind this system. The first was where a company could lease
the hardware display and data feeds they wanted from me and we would install it
and make sure that the right information displayed at the right time. The second
option was where we would make an agreement with establishments, such as major
chains like sports bars, casual restaurants, airports, etc, to deliver the
hardware and data feeds, but keep the rights to sell advertising space both on
the side of the physical devices as well as on the displays themselves, such as
scrolling text and bitmaps across the ticker hardware. At the time, this was the
most complex system I was working on, so I figured it would be fun to take a
look at how things might have changed in the past three years.
The first issue to be solved is that different data providers send data in
different ways, so abstracting out the way data is received and processed in the
system is always a good practice. I was testing out a few data provider systems
at the time, and they generally delivered their data in one of three ways:
- Over satellite linkup. This was pretty cool, except that it was a pain to set up,
and I ended up not going in this direction because it introduced a whole set
of additional physical complexities. Basically, you need a dish pointed at the
sky, and it fed data down through a cable to a converter box, and that fed out
an RS-232 (serial) feed that you could read in from a COM port.
- Over HTTP pull.
This was where you just poll a Web server for data at a given interval (some
places only allow one request every 5 minutes) and then parse it, which could
either be as a “comma-separated-value” (CSV) file or any of a bunch of others.
CSV files were easy to parse because the rows were separated by newlines
(‘\n’) and then each column was separated by commas (‘,’). I must have written
the 10 lines of code to do this a hundred times in test and trial apps.
- Over FTP push. Here you give the data provider an FTP
account, and they push data out to your server whenever it happens. The data
was usually delivered in the form of a “SLUG” which means that there was no
data in the file, but rather the filename itself contained the encoding, such
as “0001NHLNJDNYR002004F” which would decode to mean that this was the first
message of an event (usually you get updates every few minutes with the value
represented by the first 4 digits incremented each time) from the NHL game
where the New Jersey Devils were defeated by the New York Rangers by a score
of 4-2.
I ended up opting for all FTP systems because they were very easy to handle
and required little maintenance or overhead effort. I put a pretty simple system
in place where I had an app that ran in the background on the machine and would
check each upload directory every sixty seconds for new files. If new data had
arrived, it would read the data and put each set into the appropriate CSV file,
so whoever was consuming the app would only have to deal with the CSV file. The
CSV files each had different formats, depending on the type of data and the
subsets within. For example, an NHL line might look like “NHL, NJD, NYR, 2, 4,
F” whereas a golf line might look like “PGA, Woods, -1, 72, Jones, 0, 72,
Norman, 2, 71”. I knew I would eventually migrate to a database where each sport
would have a different table, but for the meantime this was very easy to work
with and useful to test before I had any live feeds.
Client requests would come in through the Web server, which seemed like the
easiest way to handle them. There were two services provided:
- The data service. This was an interface designed for client machines to
connect up and make requests through the query string of the HTTP request
(including user credentials), and a CSV file would be returned with the
requested data. It wasn’t designed to have a human interface, so CSV was good
enough and made it easy to go cross-platform. Ironically, the HTTP effort put
in on both sides was a good portion of the overall development time, whereas a
built-in SOAP Web services interface would have made my life incredibly
easy.
- The administration service. This was a simple Web UI
designed for system administrators to be able to define the playlists that
would run on each display at each location. A “playlist” was the set and order
of data to be displayed at each client site. For example, some sites might
subscribe to financial data, so they would have a playlist that would be like
“1239, NASDAQTOPGAINERS, NASDAQTOPLOSERS, NASDAQTOPVOLUME” whereas a sports
bar might have “3437, MLBNL, SPORTSAD1, MLBAL, SPORTSAD2, NHL, SPORTSAD3, NBA,
LOCALAD3437” that has sports info intermixed with paid advertisements. It also
provided a way to add and remove users, as well as define which advertisements
would be played at which venues. For example, it would be much more effective
to play beer ads at a sports bar and business ads in an airport, so this was a
key part of the service.
The biggest pain with developing these services was that I was using C++ with
the standard CGI interfaces available. For those who haven’t had the joy of
working in this environment, you would write a C++ app that would pull in the
query string from an environment variable (the oft-forgotten 3rd variable in
your main function) and then parse it, de-URL-encode it, and then do what you
need to do in the app itself, being sure to write out the HTTP response header
and the data (and you damn well better remember to put a blank line between the
header and the body or else you will be in a world of pain).
There were two intended clients for this system: the administration client
and the driver client. The administrator client used a Web browser interface, so
that was pretty easy to write off as server work (from the administrator service
above). I would have liked to have a rich client version instead, but it wasn’t
a priority item since most of the administration could be done by hand in
editing the user configuration file, which held all of the playlists in CSV
format. There was also a directory where ads were kept, but this was pretty
simple to update through FTP.
The actual driver client itself was where the majority of work needed to be
done. It was an MFC app written in C++ that would poll the server with its
credentials (every location had different credentials in order to easily
customize the playlist) and then pull down the customized playlist, which it
would parse and pass off to some core logic that would decide how to display
data. For example, an NHL score might be displayed on a scrolling ticker such
that “NHL” was in amber (yellow) on the top row, then the team and score would
be lined up as a box score with the winning team in green and lowing team in
red, and finally the game status in amber on the bottom row. Once the display
format was built, it would be passed off to the hardware driver that knew how to
transmit the proper protocol for the given hardware display, usually over serial
cable, although sometimes over TCP/IP.
Here’s an example of how the whole system looked:
Today, I would do a few things a little
differently:
- I would use C# and the .NET Framework for everything. The fact that I can write everything
in C# using the same base class libraries and APIs everywhere would make my
life incredibly easy. C# is also a much easier and more productive language to
develop with than C++ (for just about everything I was doing) which is largely
due to the .NET Framework.
- I would use Windows Server 2003 as my backend. Instead of using FreeBSD APIs to build the
file watching daemon, I would build a Windows Service that takes advantage of
the “FileSystemWatcher” object that raises events when something was uploaded.
I would also use WMI or performance counters to debug and monitor the
application, as well as event logging to keep track of anything else (or just
make sure all was well).
- I would use ASP.NET for building the Web interface. Instead of the CGI APIs to write the Web
server services, I would build a set of ASP.NET Web services that would
directly expose Playlist objects and administrator functionality. Although I
would initially wrap the administrator Web services with an ASP.NET Web app
until all of the other priorities were met, it would eventually give way to a
smart client version that consumed the same services.
- I would use SQL Server 2000 instead of the CSV files. Duh.
- I would take advantage of Windows Forms for the client
apps. I would build the administrator
app as a Windows Forms app that would use backend Web services to persist any
changes. Both apps would be delivered via no-touch deployment, making it
really easy to keep everything up to date without having to worry about
individual updates to individual machines. The integration would be done
entirely through Web services, which is pretty much automatically handled by
Visual Studio and the .NET Framework without any coding. All of this means
that I would have to write 0—yes ZERO lines of deployment-handling code and
ZERO lines of integration-handling code. That would probably cut out 60% of
the overall source investment. Also, since SOAP provides loosely-coupled
integration, I wouldn’t have to worry about the versioning of components if I
changed or added anything on the server as long as the same endpoints existed
across versions.
- I would consider making a special Windows XP Embedded client to
replace the driver PC. This could go either way, but it might just be
easier to build a lightweight hardware attachment running XPe with the .NET
Framework, serial and network drivers, and have the apps included, which would
plug into the side of the display hardware. This would result in a very
specialized system (which might not even require UI) and be much easier to
conceal in the deployment environment.
My new system would probably look something like this:
I really miss the world of hands-on software development. Thinking about all
of my old projects has made me very nostalgic, and I must admit that I’m a
little jealous of everyone who gets to actually write code these
days.
I'm at my second hotel in the Boston area this week and
they've both provided complimentary high speed. This is a trend I could get used
to.
Check out GDN Workspaces if you
get a chance. Some cool new stuff:
- Visual Studio integration with source control!
- RSS feeds for news and releases!
- Ranking features for workspace searches!
I'll be enjoying this once I get a chance to dive back into Dimensions.
By the way, I'm pretty sure "GotDotNet Workspaces v1.0 Is Awesome" is a
grammatically correct title because "GotDotNet Workspaces" is a singular
entity...right?
I had a meeting with Chris Anderson, one of the masterminds
behind several popular products including BlogX and Windows. I haven't been this
starstruck since Ronald McDonald made a surprise appearance at the Bellevue
Square Mall.
I just found out that "disco fever" is not covered by
Microsoft Health. I guess I should stop walking into meetings with "Stayin'
Alive" blaring on my tablet speakers.
I spent over six hours in a car with Mike Kass this
weekend driving back and forth to a cabin in Olympic National Park. For those of
you who don't know Mike Kass, picture Skippy from Family Ties, and then give him
a +2 in "niceness". Mike is a great guy, but if I have to listen to the
soundtrack to Saturday Night Fever one more time I might lose it. The one thing
that kept me sane is that he does a great Bee Gees impression (and also Axl
Rose, but that's for another blog entry).
Seriously.
I had been planning to propose to my girlfriend during the month of June
(usually the best time of year in Seattle). Unfortunately, it turned out that I
had taken on a last minute trip that would keep me out for the rest of the
month, so my opportunities were pretty limited. I figured I'd have to make the
best of it and knew I could meet the challenge of finding a unique way of
surprising her.
The first step was to throw her off guard with my carefree attitude towards
engagement. I had taken her out to look at rings a few times so I could get an
idea of what she liked, but always played it off as a "someday" thing. We got to
a point where she realized that it was going to happen, and it was pretty much a
matter of trying to figure out when I was going to pop the question. However, if
the topic came up in conversation, I would say things like "I was at [local
supermarket] Fred Meyer today and saw their jewelry department…they were
right--I couldn't tell the difference between the real and fake diamonds" or "I
found a ring shop that will mount a nicely broken piece of windshield glass for
only a few bucks". I also told her about a website I had found where they have
Word templates for diamond certifications, so I might be able to certify my own
“diamonds” with the help of a color printer and some lamination magic.
After that I set up a bunch of possible occasions to keep her guessing. On
Wednesday she found out that she had a last minute one-on-one meeting with our
senior vice president, Eric Rudder, (she also works at Microsoft) so I told her
that we would go out to “celebrate” afterwards at the Space Needle for dinner
(which was an obvious “tip-of-the-hand” to her). Then again, we were also going
to spend this weekend at an oceanfront cabin, so maybe that would be it. If not
that, she would be flying to meet me in Boston next weekend for a friend’s
wedding, and if it didn’t happen then she figured that spending the 4th of July
in my hometown of New York would be an absolute guarantee. As a result, she
spent half this week trying to figure out where and when I was going to make my
move, and the other half preparing for the Eric review.
I’d like to take a few moments out to talk about an “Eric review”. These
reviews, like pretty much all executive reviews, are great challenges and
“rights of passage” that all employees get to go through at one time or another.
There are tons of urban myths about Microsoft executives tearing people to
pieces in reviews, so the “first time” can be very intimidating for an employee.
The reality is that the executives expect everyone to feel a strong sense of
ownership of their responsibilities and when people are unprepared or careless
with decisions they can be reminded of their responsibilities in a very direct
way. In fact, Microsoft was the first place that I had ever heard the term “own”
used to describe a role, such as “I own product planning for X” or “I own
performance testing on X”. I have realized that using the term “own” is a great
thing because it truly describes the level of responsibility and passion people
need to feel for what they do. If I “own the developer experience for building
applications using ABEK”, I had better know the top feature requests from ABEK
developers, the top improvements we’re making in the next version (and have damn
good reasons if the two lists don’t match exactly), the best ways to communicate
with ABEK developers, the top reasons why non-ABEK developers don’t use it, etc.
As you can probably imagine, it takes a lot of work to keep on top of all of
this, so reviews often require a lot of preparation. If you’re not prepared then
you’re not owning your responsibilities and you’re wasting everyone’s time.
So why did I bring all of this up? Well, I wanted to help you get a good feel
for the amount of effort she put in to getting ready for this review. It was
hours upon hours, and I spent a lot of time with her helping her to craft
wording for slides, as well as sit through hours worth of practice runs. It was
very time consuming for me, but well worth it.
How was it “well worth it” for me? Simple. It was well worth the opportunity
to spend the time with her. It was well worth the gain in knowledge of her
expertise. It was well worth the perspective she has with reviews. However, more
than anything, it was well worth it for the look on her face when she entered
the conference room for the review and realized that Eric wasn’t there—I was.
You see, there really was no Eric review. We had set it all up with the help
of some great people on Eric’s team. She walked in with a slide deck and out
with a ring—and a story.
In the end, that’s all she really wanted.
I just got engaged. More to come.
I'm going to be in Boston from Wednesday, June 25,
through Friday, June 27. If you work as part of a
development team and have some free time in your schedule, please let
me know if you'd like to meet for an hour or two to talk about the stuff you do.
I'm interested in learning about the types of applications your team
works on, the tools and processes you use, and anything about whatever you think
Microsoft should build, fix, or improve, even if you're not using anything
from Microsoft to begin with. Any information you share can be
held in confidence with Microsoft and just used to build better products to meet
your needs.
If you're available and interested, please ping me at EdKaim@microsoft.com.
More Posts
Next page »