Setting up a robust infrastructure: Are Cisco stuck in the 90'ies?

Beeing an entrepreneur really exposes you for all levels of operational handywork. The one day I'm putting the finishing touch on the businessplan and the budgets, and the next I'm stuck in the server room trying to get the firewall running. Now, I wouldn't normally blog about infrastructure. But this topic also applies to general attitudes in the software industry, and provides an example of what I see as customer hostile behaviour which causes losses for all parties in the value chain. So; if you're up for a little rant, read on.

Usually I'm relying on the principle of comparative advantage for tasks such as hardware firewall and networking stuff (i.e. I'll do the development, someone else is better suited to set up my firewall), but when in a startup I really feel that I need to be proficient on something as important as my core infrastructure.

I've done some pretty heavy research on how to set up Puzzleparts infrastructure topology to support some primary goals:

• Very high flexibility from Windows 2008 Hyper-V virtualization
• Comply with best practices for Microsoft Windows based topologies
• Mimic common infractructure setup for future Puzzlepart customers to enable rigid testing in a realistic environment and dogfooding
  
• Complete control over security

Now, in theory all of this could have been accomplished with Microsoft server software, but after liaising with a few of my favorite Microsoft IT Pros the message was pretty clear: Having a firewall appliance as the edge border is both wise and recommendable. This is even suggested in ISA Server training out there (see video training here and here).

So a pretty standard setup would then be a Cisco ASA 5505 in front, backed by a ISA Server for virtual application routing (routing subdomains to differend webservers in the backend). I got my Cisco ASA 5505 box last week and started reading up on the manual. My primary goals for the Cisco box were simple:

• Use it as primary firewall and only exposed point to the internet
• Establish a private subnet behind the Cisco ASA 5505 to allow easy VM instantiation
  
• Route webtraffic from Cisco ASA 5505 to dedicated webservers (ISA Server which routes further)
• Allow for Cisco AnyConnect secure VPN connections against the Cisco ASA 5505 to allow for full "LAN feeling" with SSL VPN connection

After reading the manual front-to-back and researching online, these goals seemed to be quite easy to accomplish. I went ahead and created a Infrastructure diagram and a sensible IP addressing plan for the Puzzlepart servers.

Then I headed out for the server room. After a couple of hours struggeling with the initial setup, following the manuals step by step, I realized that the installed software on the Cisco ASA 5505 was far from updated. I did a cross check against the supplied CD and found that it contained more recent versions of several elements of software. The important ones for my setup was:

• The core ASA software (the ROM'ish os software for the appliance itself
• The ASDM management software which provides a complex UI for managing the box
• The AnyConnect VPN software client for SSL VPN (the traditional IPSec client for Cisco doesn't and won't ever support Vista)

Afraid to do something wrong in the upgrade I turned to Cisco and their upgrade guide for the ASA software first. This instructed me in uploading a new image to flash and also upload the updated ASDM (management sw) to flash.

A new boot image required a reboot of the ASA firewall. However, when I tried access the Cisco ASA 5505 again the ASDM managment util couldn't connect due to incompatible version between the ASA boot sw and ASDM software (both from same supplied cd). This headed me towards cisco.com for a quick download of a newer ASDM. Problem was:

Cisco doesn't allow for download of their software without a service agreement. 

Now, that WOULD be okay if they actually shipped the boxes with functional and up-to-date software. But they don't. I had to spend more than 6 hrs on the phone in order to get my hands on a newer ASDM version. A newer ASDM (not newest, and not the rest of the required stack to run the box) was provided "unofficially" by Cisco personell, which were most helpful; but they made it very clear that "this was not official, and that I DID have to buy a service agreement to get the current software" (!?!)

The new ASDM version was still not the latest at the time of the purchase, but at least it slightly worked. A lot of quirks there; especially with the Cisco "Apply/Save" mechanism for making config changes which seemed very flaky. Now for another pile of hours to get VPN running. Also here I followed various guides from cisco (the best one seemed to be this one, even though it doesn't cover AD authentication). A good tip for any of Ciscos guides is to make sure you're reading the guide for your version of their software. There are significant differences.

After another frustrating chunk of hours everything was up and running except Internet connections for VPN clients. Now this should be open and shut: Split Tunnelling was configured and only local VLAN addresses were configured to be tunnelled. When it just didn't work I brought in a friend with a lot of Cisco experience, but we couldn't crack it.

Why? There is a bug in Cisco AnyConnect < version 2.1 with Split Tunnelling under Vista. In other words: I'd just spent 4-5 hours (x2, i brought my Cisco friend with me) for something that just should have worked if Cisco would supply me with their latest BUGFIXED software at the time of the purchase.

In this whole process almost anyone I talked with (most with wast experience in working with Cisco) were very negative to Ciscos policy on software distribution. For my own experience I would have been better off buying a box twice the price PLUS a consultant for setup. And the most annoying part is: If Cisco would just have allowed CUSTOMERS (yeah, that important group in your business model) to get to the latest software (read: least-buggy) at the TIME OF THEIR PURCHASE none of this would have happened, and I would be writing this flame.

So to you guys at Cisco, including CEO John Chambers (who just spent 8 pages in HBR talking about how incredible attuned they are to new trends):

Stop hogging your software. Allow us to fix your problems with your software easily by allowing us (customers) to get most recent software. Make me WANT to buy a service agreement, don't make it feel like an additional penalty just after we've shown you my confidence in buying your product. And: don't treat small and SOHO customers with a Enterprise attitude, that just gives you a bad rep.

Looking forward to seeing changes in your practices in the future, but I'm afraid I'll prefer other vendors next time.