Jon Galloway

Troubleshooting an Intermittent .NET High CPU problem

We’d been getting sporadic reports of high CPU usage in Witty (a WPF Twitter client). I’d tried running the application in debug mode for a while and could never get it to occur, but finally I saw it happening while I was running a release build (keep in mind that 53% is indicating that one of my two CPU cores was saturated):

Great, let’s attach a debugger to see what’s going on:

Troubleshooting high CPU use often leads to chasing heisenbugs, since the CPU usage of course drops when you break into the code. But, I could at least step through the current code a bit to get an idea of the current state. Doing that showed a lot of time at System.Threading.WaitHandle.WaitOne, which made me suspect at threading issue.

On the other hand, breaking into a multithreaded app could have caused that, too. We need more information – let’s take a look at the current threads:

Note to self: should have named the threads so I’d have a better idea of what they’re all doing. I’ll talk more about that in a little bit.

Now, there are probably two ways to go from here:

• Smart, systematic debugging
• Brute force hacking

In this case case, since we don’t have that many threads to deal with, I think starting with some brute force interrogation is a good way to get started. If we don’t find anything, we’ll cool off and do the whole Sherlock Holmes contemplation thing, complete with pipe and violin.

First Guess: The Red Herring

This is not the real problem, it’s a wrong guess. I’m including this for two reasons:

1. It’s the truth. It’s what I really did. I’m not going to edit out my stupidity… this time.
2. It shows a little interesting information on loading symbols from the Microsoft symbol servers to step into and debug .NET framework code

That said, if you want to skip over this bit, I won’t be offended in the least.

So at this point, I started just looking through the thread list. When we activate a thread, even in disassembly, we can get a pretty good idea of what it’s doing. There’s a current call stack, and the disassembly view shows us the current executing method:

Well, our current hunch is that the System.Threading.WaitHandle.WaitOne call has to do with the problem, and it’s showing up as the current executing method, so it’s worth looking at for a bit. But that disassembly is over my head. We need to get the actual source… fortunately, that’s not tough. We can load the source from the Microsoft Symbol Servers:

And a few seconds later, we’re interactively debugging framework code. Paging Dr. Awesome…

Note: If you’re not able to see the framework code in the call stack, you’ll need to turn off the “Just My Code” debugger option:

I’ll spare you the embarrassment of watching me wander around the WPF code which pulls down databound images from internet addresses. It’s interesting code, but after poking around for a bit I convinced myself that this wasn’t the source of the high CPU issues.

Wising Up: Freezing Threads To Find The Culprit

At this point, I realized that I needed to get a little smarter. Instead of just browsing through all these threads just hoping to find the problem, I needed a smarter way to determine which one was actually the problem. Well, how about this?

1. Break into the debugger
2. Freeze one of the threads
3. Resume the app
4. See if the CPU usage has dropped – if so, we’ve found the bad thread, so we can debug it
5. If not, repeat

Simple enough. You can right click any of the threads in the list and freeze it:

Now, this is still kind of voodoo troubleshooting at this point, since there’s a possibility that the problem’s caused by the interaction of two different threads, or something else that’s going to be masked by a one-at-a-time approach. I’m playing another hunch here – I know enough about this application to assume that most of the threads are independent.

Intermission: Play Along At Home with this Simple Runaway Thread Demo

This may be a little tough to follow since there are a lot of moving parts. Let’s step back and look at a very simple application with one runaway thread. Create a new console application and add the following code:

using System.Threading;

namespace RunawayThreadSample
{
    class Program
    {
        static void Main(string[] args)
        {
            for (int niceThreads = 1; niceThreads <= 10; niceThreads++)
                new Thread(() =>
                   {
                       while (true)
                           Thread.Sleep(500);
                   }) { Name = "Thread " + niceThreads }.Start();

            new Thread(() =>
               {
                   while (true)
                   { }
               }) { Name = "Evil Thread" }.Start();
        }
    }
}

A few things to notice here:

1. We’re naming our threads this time. With .NET 3.5’s object initializer syntax, we can set the name when it’s created.
2. We’re using lambda syntax to define the anonymous thread delegates – that’s the “() => {…}” stuff. Karl Seguin covered that pretty well in his Back To Basics series.

Start up Task Manager (or run ProcMon), then run the application. You should see your CPU use pegged at around 100%, as the “Evil Thread” spins in a tight loop. Now break into the debugger – you should see your CPU usage drop as all the threads are suspended. Freeze the Evil Thread (right-click, freeze):

Then resume the application (by hitting F5) and you should see your CPU usage stays low.

Returning To Our Regularly Scheduled Program…

Sure enough, freezing and thawing threads in turn points me to a single thread which is causing the high CPU usage – one that’s spending a lot of time in Witty.SingleInstanceManager.WaitForSignal(). Interesting… a little searching lets me know that this class came from a CodeProject article about how to enforce a single instance for an application. So that explains why this bug was hard to reproduce – it only happened when the application was already running and you tried to start a second instance. That’s not something you’d normally do on purpose, so the bug just appeared to happen randomly.

But why didn’t we ever see it in debug mode? Let’s look at the code that sets up the single instance manager:

/// <summary>
/// Enforce single instance for release mode
/// </summary>
private void SetupSingleInstance()
{
#if !DEBUG
    Application.Current.Exit += new ExitEventHandler(Current_Exit);
    _instanceManager = new SingleInstanceManager(this, ShowApplication);
#endif
}

And there we go – the single instance manager only runs in release mode. Heisenbug magnet! Removing the #if pragma command lets us debug the single instance manager, so now it’s time to take a look at that WaitForSignal() method:

// thread method will wait on the event, which will signal
// if another instance tries to start
private void WaitForSignal()
{
    while (true)
    {
        uint result = WaitForSingleObject(m_EventHandle, INFINITE);

        if (result == 0)
        {
            m_Instance.Dispatcher.Invoke(System.Windows.Threading.DispatcherPriority.Normal, m_callback);
        }
        else
        {
            return;
        }
    }
}

There’s the bug - if you know the answer, just keep quiet so you don’t spoil it for the rest of the class.

Let’s take a look at how that WaitForSingleObject thing is working. First, when the application starts, we’re grabbing an Event Handle with OpenEvent, then spinning off a thread that calls WaitForSignal (code above):

public SingleInstanceManager(Window instance, ShowApplicationCallback callback)
{
    m_Instance = instance;
    m_callback = callback;

    //try to our event
    m_EventHandle = OpenEvent(EVENT_MODIFY_STATE | SYNCHRONIZE, false, EVENT_NAME);
    if (m_EventHandle == IntPtr.Zero) //if it doesn't exist
    {
        //create our event
        m_EventHandle = CreateEvent(IntPtr.Zero, true, false, EVENT_NAME);
        if (m_EventHandle != IntPtr.Zero) //if successfull
        {
            Thread thread = new Thread(new ThreadStart(WaitForSignal));
            thread.Start();
        }
    }
    else
    {
        SetEvent(m_EventHandle);
        MessageBox.Show("There is already an instance of Witty");
        Environment.Exit(0);
    }
}

So, here’s how it’s working – the OpenEvent is getting a handle, and the WaitForSignal method calls WaitForSingleObject, which will return 0 when signaled (that is, when another instance of the same handle is detected). Note – it looks like we’re running a tight loop with while(true), but when WaitForSingleObject is called with an INFINITE timeout, it won’t return until the object is signaled, meaning that the loop will never complete a single run unless a second instance is run.

But what happens when a second instance is started? Well, WaitForSingleObject returns 0, and we invoke the callback method, which just shows the application. No problem…

Except that now we’re in a tight loop. We’ll complete the loop, call WaitForSingleInstance, which will now immediately return 0 because it’s been signaled. There’s our high CPU bug. The solution is to call ResetEvent once we’ve shown the application, so that WaitForSingleObject won’t fire off again the next time through:

// thread method will wait on the event, which will signal
// if another instance tries to start
private void WaitForSignal()
{
    while (true)
    {
        uint result = WaitForSingleObject(m_EventHandle, INFINITE);

        if (result == 0)
        {
            m_Instance.Dispatcher.Invoke(System.Windows.Threading.DispatcherPriority.Normal, m_callback);
            ResetEvent(m_EventHandle);
        }
        else
        {
            return;
        }
    }
}

Now that we’re calling ResetEvent, we stay out of that tight invocation loop and we’ve licked this CPU bug. To be honest, unnecessary loops are a common source of bugs, and this one still kind of freaks me out even if the timeout on WaitForSingleObject is infinite. Given given time I’d like to look at some other single-instance solutions, but there’s no real performance driver for that.

Just Hacking Here

This has pretty much been “Git ‘R Done” debugging. It’s hacking. Hopefully it’s helpful to you, but I know that there are folks out there with some real skill at diagnosing application performance issues, and there are better debugging tools available, too. How would you go about diagnosing something like this?

Windows 7 Betta

I’ve been running Windows 7 Beta 1 for a week now and really like it. But I’d been looking at desktop for a few days before someone pointed out the little “desktop Easter egg”.

Here’s the desktop:

See it? Kirstin Juhl pointed it out on Twitter:

OK.... I just "got" the fish on Windows 7... it's a beta fish. I never would have put it together.... my 5year-old saw the screen and asked, "Is that a beta fish mom?" I looked and said "Ohhhh!!!"

Technically, it’s a betta fish, but I’m not going to quibble about that. So, enjoy the betta!

Time released content in ASP.NET

While working on the PDC2008 website, we had several time-critical updates. There were some announcements that needed to go live on the website at specific times to coincide with other marketing, there were updates to the list of of software being given to attendees that needed to go live right after the keynotes in which they were announced, etc. While some of the site ran on RSS feeds, on some pages we needed the flexibility of static HTML and CSS. While there were plenty of times where I made that sort of deployment by hitting upload in Filezilla at just the right time, there were other times where that wasn’t possible.

The first time that hit was in August, several months before the conference, when we needed a site update to go live while I was scheduled to be in the middle of a flight. Clearly, a technological solution was needed.

Extending The ASP:Placeholder Control

Yes, all you ASP.NET MVC hipsters can wander off now – this is an ASP.NET Webforms solution, since the site initially went live when ASP.NET MVC was at Preview 3 (5 months before hitting Beta). But – wait – please leave a comment on how you’d approach this with ASP.NET MVC, would you?

The ASP.NET Placeholder control is a simple control that renders no markup of its own, it just holds content. It’s great for holding blocks of content which is either displayed or hidden, among other things. So in this case, I inherited from Placeholder and created TimedContentPlaceholder, adding a Start and End property. That let me do this kind of thing:

<PDC:TimedContentPlaceholder runat="server" Start="10/28/2008 10:30 AM">
<% /* 
    Phase 3 (to go live Monday, October 28 – following Keynote 2) : Please add 
    as a block at the top of the page under a heading reading, "BITS UPDATE 02"
*/ %>
<h2>BITS UPDATE 02</h2>
    <b>Windows 7 Ultimate ISO</b></h3>
<p>
    Burn your own Windows 7 Ultimate DVD using these 32bit &amp; 64bit ISO files.</p>
<PDC:TimedContentPlaceholder>

So that content needed to be prestaged, but shouldn’t go live until the specified date and time. Here’s the control which handled that:

public class TimedContentPlaceholder : System.Web.UI.WebControls.PlaceHolder
{
    public DateTime? Start { get; set; }
    public DateTime? End { get; set; }
    public Uri Redirect { get; set; }

    public override bool Visible
    {
        get
        { return base.Visible && isInTimeRange(); }
        set
        { base.Visible = value; }
    }

    protected override void OnInit(EventArgs e)
    {
        if (Redirect != null)
            if (isInTimeRange())
                Page.Response.Redirect(Redirect.AbsoluteUri);
    }

    private bool isInTimeRange()
    {
        if (Start != null && Utility.GetCurrentDateTime() < Start)
            return false;
        if (End != null && Utility.GetCurrentDateTime() >= End)
            return false;
        return true;
    }
}

Redirection

Most of that’s pretty straightforward, but there are a few extra things there. First of all, there’s the Redirect property. That’s useful when you’ve got a basic page in place which will later redirect to an application or even another server later. For instance, we used that to redirect to the server hosting the session information on the cutover date using something like this:

<PDC:TimedContentPlaceholder runat="server" Redirect="https://sessions.microsoftpdc.com/" Start="8/15/2008 12:00 AM" />

Time Machine

The second thing is a bigger deal – we weren’t just using DateTime.Now in our time checks, we called out to a function. There were two reasons for that – the simple reason was to provide a central place to handle the timezone conversion between the conference location and the server location. But the main reason was to allow specifying a test time, which allowed us to see how the site would look at any time by passing in querystring arguments. Here’s how Utility.GetCurrentDateTime() worked:

public static DateTime GetCurrentDateTime()
{
    //Compensate for server being on the east coast
    if (IsProduction())
        return DateTime.Now.Subtract(new TimeSpan(3, 0, 0));

    DateTime testDate;
    if (HttpContext.Current.Session["date"] == null)
        return DateTime.Now;
    if (DateTime.TryParse(HttpContext.Current.Session["date"].ToString(), out testDate))
        return testDate;
    return DateTime.Now;
}

There was separate code in the masterpage which took a querystring variable and set a session variable with the test date. We also pulled that session variable in Silverlight elements as well, so we could step through the site evolution as the conference approached, each keynote completed, and the conference ended.

Not rocket science, but it worked for us. How would you have solved this?

Silverlight Crossdomain Access Workarounds

I was testing out some typography with Silverlight and figured I’d try grab some text from Wikipedia. I started with the naive approach:

private void GetText()
{
    WebClient webClient = new WebClient();
    webClient.DownloadStringCompleted += new DownloadStringCompletedEventHandler(webClient_DownloadStringCompleted);
    webClient.DownloadStringAsync(new Uri("http://en.wikipedia.org/wiki/george_washington/"));
}

void webClient_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{

    //This next line throws an exception 
    string fullText = e.Result;

    //Do some neat stuff...
}

That throws a pretty ambiguous exception: 'e.Result' threw an exception of type 'System.Reflection.TargetInvocationException'. The InnerException just shows “Security Error.” The source of the Security Error is that Wikipedia doesn’t have a Crossdomain ( crossdomain.xml) policy file.

Best Case: Site supports crossdomain.xml or clientaccesspolicy.xml

A lot’s been written about the Site Of Origin policy for both browsers and RIA clients like Silverlight and Flash, so I won’t rehash them in too much detail. The idea is that, for security reasons, code running in a webpage (Javascript, Silverlight, or Flash) should generally only be able to access the domain that hosts the webpage. Silverlight followed Flash’s lead and allows for cross-domain calls if the site it’s accessing has a crossdomain.xml file which permits it. This doesn’t really protect against cross-site scripting (XSS) attacks, since – as Rick Strahl noted – any malicious site can post a crossdomain.xml file (e.g. http://evilsite.com/crossdomain.xml) and RIA’s will then be allowed to pass information home. Rather, this is a safeguard against cross-site request forgery (XSRF) attacks. I think Steve Sanderson’s comment sums this up pretty well:

What you're missing is the CRSF scenario. Current browsers prohibit me, from www.pirate.com, using script to cause a GET or POST request to www.bank.com and actually reading the response data back in my script that I host onwww.pirate.com. This is important because requests sent to www.bank.com will include the cookies for that domain and may therefore be treated as authenticated requests, for which the server will return sensitive information. You don't want that sensitive information to be retrievable by the script running in www.pirate.com.

This […] allows cross-domain requests to read data across domains *only* if the target domain opts-in on a *per-request* basis. Obviously, www.bank.com isn't going to opt in for any sensitive URLs, but it might choose to do so for some kind of non-sensitive API (stock quotes or whatever).

So, crossdomain protection means that I can use a potentially evil Silverlight application without worrying that it’s going to use my browser context to try to access my bank or e-mail or other trusted data, because my bank isn’t going to allow cross-domain access. So evilsite.com can “phone home” with any cookies it has access to, but it can’t use my authenticated status to make requests to my bank.

Tim Heuer has written several good posts on cross-domain issues. I especially recommend his crossdomain.xml helpers for Visual Studio if you need to write your own crossdomain.xml file to expose your own services.

The Silverlight Web Services Team blog has some good troubleshooting tips in case you’re having trouble connecting to a site which has a crossdomain.xml file, and you can use this simple Silverlight Cross Domain Policy File Checker to quickly check sites. My top recommendations:

1. Make sure you’re not testing from a file:// URL, since websites running under file:// can’t make cross-domain calls. You need to test with an http:// page – for instance, using a test ASPX page running in the Visual Studio WebDev.exe test server.
2. Browse to the crossdomain.xml file (e.g. http://twitter.com/crossdomain.xml) to make sure that it’s there.
3. Test in Fiddler, Firebug, or the IE Web Developer Toolbar to watch network traffic and verify your Silverlight app is trying to connect to the right URL for crossdomain.xml.
4. Build a very simple test Silverlight app which does nothing but make the cross-domain call.

Note: clientaccesspolicy.xml vs. crossdomain.xml

Silverlight supports a subset of Flash’s crossdomain.xml file format, but doesn’t allow for fine grained control. If you’re exposing your own services to Silverlight, you’ll want to look at clientaccesspolicy.xml; if you’re supporting Flash and Silverlight you should ideally have both. However, in this post I’m concentrating on accessing external resources, which at this point are more likely to have crossdomain.xml due to the fact that it works on Flash as well.

Second Best: JSON Requests

JSON requests get a free ride – they can connect to any any site, regardless of whether they have a crossdomain.xml file. That’s probably because JSON is just an object notation (name/value pairs and ordered lists), rather than direct script. While it can be used for evil (via the evil eval()), it’s more of a serialization format than a script. Regardless of the reasons, it’s allowed to cross domains when other calls can’t.

Joshua Allen recently wrote about making JSON requests from Silverlight. His method depends on some Javascript methods being included in the host webpage, but it’s possible to append the Javascript to your DOM from Silverlight if you’re writing an embeddable widget (say, for instance, for the MIX09 10K Challenge).

Read Joshua’s post for the full details, but the basic idea is that you have an injection script in your page, like this:

function injectScript(url) { 
   var head = document.getElementsByTagName(‘head’)[0]; 
   var script = document.createElement(’script’); 
   script.type = ‘text/javascript’; 
   script.src = url; 
   head.appendChild(script); 
}; 

Then we invoke the injection script from our Silverlight application:

HtmlPage.Window.Invoke("injectScript", url);

The script source needs to point to a URL that returns JSON, and preferably wrapped in a callback function call, like this:

callback({"info": "important info here"}

Then (continuing to quote Joshua Allen’s example), the Javascript callback calls into a scriptable method in our Silverlight application:

function callback(obj) { 
   var silverlight = document.getElementById("silverlight"); 
   if (silverlight) { 
        silverlight.Content.Page.PassData(JSON.stringify(obj)); 
   } 
};

Note 1: About that JSON object

The call to JSON.stringify() is natively supported in IE8, Firefox 3.1, and is slowly working its way into Webkit (Safari, Chrome). But it’s easy to add JSON for other browsers by using the parser distributed on json.org: http://www.json.org/json2.js

Note 2: About that Callback

By convention, most API’s that return JSON will take a parameter for the name of the callback function. For instance, here we’re telling the html-whitelist service to call mySuperFunction with the results:

http://html-whitelist.appspot.com/whitelist?content=%3Cscript%3Escary+html%3C/script%3E&json=1&callback=mySuperFunction

Flickr defaults to jsonFlickrApi, but allows you to override the callback name with &jsoncallback=muSuperFunction

http://api.flickr.com/services/feeds/photos_public.gne?tags=silverlight&lang=en-us&format=json&jsoncallback=mySuperFunction

returns this (only showing the first image):

mySuperFunction({    
        "title": "Recent Uploads tagged silverlight",
        "link": "http://www.flickr.com/photos/tags/silverlight/",
        "description": "",
        "modified": "2008-12-11T19:22:44Z",
        "generator": "http://www.flickr.com/",
        "items": [
       {
            "title": "NEC Biglobe album viewer",
            "link": "http://www.flickr.com/photos/adamkinney/3100209567/",
            "media": {"m":"http://farm4.static.flickr.com/3294/3100209567_43d738e294_m.jpg"},
            "date_taken": "2008-12-11T11:22:44-08:00",
            "description": "description here…",
            "published": "2008-12-11T19:22:44Z",
            "author": "nobody@flickr.com (adKinn)",
            "author_id": "83775906@N00",
            "tags": "album silverlight deepzoom necbiglobe"
       }
        ]
}) 

Note 3: What to do with that JSON data?

In this sample I’m just passing the JSON data back to my Silverlight application as a string. You’re not on your own dealing with parsing that string, though – the System.Runtime.Serialization.Json namespace can handle that for you. Corey Schuman’s written a nice walkthrough on how to consume a JSON object in Silverlight.

No JSON? Don’t waste your time on IFRAME / JSONP hacks

I spent some time looking into ways to pull data from a URL that doesn’t supply data in JSON format. My conclusion is that it’s a bad idea. While you may be able to sneak something by an older browser, you’re essentially hacking at this point, and newer browsers will block it.

For instance, let’s try to create an IFRAME and load the content:

public Page() 
{ 
    InitializeComponent(); 

    HtmlPage.RegisterScriptableObject("Page", this);   

    iframe = HtmlPage.Document.CreateElement("iframe"); 
    iframe.Id = "invisibleIframe"; 
    iframe.SetProperty("src", url); 
    iframe.SetStyleAttribute("display", "none"); 
    iframe.SetAttribute("onload", "contentLoaded"); // call a function which calls silverlight.Content.Page.Loaded() 
    HtmlElement body = (HtmlElement)HtmlPage.Document.GetElementsByTagName("body")[0]; 
    body.AppendChild(iframe); 
} 

[ScriptableMember] 
public void Loaded() 
{ 
    HtmlWindow content = (HtmlWindow)iframe.GetProperty("contentWindow"); 
    HtmlElement doc = (HtmlElement)content.GetProperty("document"); 
    HtmlElement body = (HtmlElement)doc.GetProperty("body"); 
    string innerHTML = (string)body.GetProperty("innerHTML"); 
    Output.Text = innerHTML; 
}

While that will cause an IFRAME to be created and load the document, the IFRAME content isn’t accessible to Silverlight (or to Javascript functions running in the page, for that matter). Makes sense, the Single Origin Policy is being enforced.

Also a no-go: using the JSON approach on a URL that’s not returning JSON. Two problems:

1. Script tags with an external source are evaluated when loaded, and if they’re not valid Javascript you’ll get an error message. For example:
   <script src=”http://google.com></script>
   will try to execute the HTML code of the Google home page as HTML and will throw Javascript errors (depending on the user’s browser settings).
2. We won’t be able to access the content that the script tag contains from Silverlight or Javascript – it’s blocked by the the Site Of Origin policy.

Are there ways to hack around this? Probably. But, I’m convinced that it’s a losing battle – if you get it to work, you’re taking advantage of a bug. Browsers are enforcing a rule, and if you find a way around the rule, you should expect that it’ll stop working at some point, or won’t work on all the browsers you want to support.

Crossdomain.xml and JSON Friendly Proxies

So, what do you do if you want access to a web resource, but the site doesn’t provide a crossdomain.xml policy or expose data in JSON format?

The answer is to use a proxy who does allow access. You can write your own, of course. I wrote some posts about that back in the Silverlight 1.1 days, but of course that showoff Tim Heuer’s gone and upstaged me with a super-nice post on how to call Amazon S3 Services from Silverlight 2 with a custom webservice proxy. And to add insult to injury, he topped it off with another post on reading data and RSS with Silverlight and no cross-domain policy. He points out Yahoo Pipes, which is a workable solution for many cases, with some important caveats:

Note on usage: The module will only fetch HTML pages under 200k and the page must also be indexable (I.E. allowed by the site's robots.txt file.) If you do not want your page made available to this module, please add it to your robots.txt file, or add the following tag into the page's <head> element:

<META NAME="ROBOTS" CONTENT="NOINDEX">

For example, I created a simple pipe which grabs the HTML from a page and returns it in any of the formats Pipes supports (including JSON):

So, we can use it to scrape the HTML from the Silverlight page on Wikipedia and get access to the HTML content from our Silverlight application like this:

http://pipes.yahoo.com/jongalloway/url2json?URL=http://en.wikipedia.com/wiki/Silverlight&_render=json

That Pipe is published and is freely accessible, so you can either use it or clone it for your own use if you’d like. So if you’re after HTML content, that approach works provided the page is indexable and is <200KB. If not, you can see if the content’s in the Wayback Machine, since archive.org has a crossdomain policy file. To get the latest version of a page from the Wayback Machine, just use this format: http://web.archive.org/URL, e.g. http://web.archive.org/http://microsoft.com. They don’t provide the content in JSON format, but you can get it via a WebClient request.

If you want access to data that’s got an RSS feed, you’ve got a lot more options. Feedburner supports crossdomain, and there are piles of Feed To JSON pipes on Yahoo Pipes, such as jsonifier. For instance, StackOverflow doesn’t (yet) support crossdomain access, but we can get a JSON feed of questions tagged with Silverlight (http://stackoverflow.com/feeds/tag/silverlight) with that jsonifier pipe:

http://pipes.yahoo.com/pipes/pipe.run?_id=NFpLzYbC2xGoZ8blE2_cUw&feed_url=http://stackoverflow.com/feeds/tag/silverlight&show=400&url_filt=__nomatch___&_render=json

and if we wanted it in XML format, we could just change the _render action:

http://pipes.yahoo.com/pipes/pipe.run?_id=NFpLzYbC2xGoZ8blE2_cUw&feed_url=http://stackoverflow.com/feeds/tag/silverlight&show=400&url_filt=__nomatch___&_render=xml

Calls To Action

1. Lobby sites which provide external data to support crossdomain.xml. Be sure to let them know that it will enable both Flash and Silverlight to access their data.
2. I’ve started a community wiki at Stack Overflow to list sites which do support crossdomain.xml. Please add to the list if you know of some I missed.
3. Write some cool mashups!

Epilogue

Oh, and wikipedia? The thing that got me started on this quest? It turns out that they do have an API that returns JSON.

Looking back at MicrosoftPDC.com (from the inside)

I had the privilege of working on the MicrosoftPDC.com website as lead developer for the past several months. The process hasn’t been kind to my blogging schedule lately, but the experience definitely taught me quite a bit: working with the top-notch Microsoft developer evangelism team, setting up a site for maximum flexibility, setting up the Silverlight experience, and troubleshooting some interesting issues during the conference. I’m going to run through several of these at a high level and may dig into some of these in more detail later (so comment if you want to hear more about something).

As I’m describing some of these, remember that while I was the lead developer, these sites were built by a team. The project and UI experience were managed by the Microsoft Developer Evangelism team, the visual design was based on the overall conference visual design concept. I lead the development team which included some of our super-awesome developers and designers here at Vertigo¹. So I’ll take some of the credit for what went well and most of the blame for what went wrong. Also, while the experiences were kept in sync, we didn’t do any of the work for the the registration and live conference session information (including video links, evals, etc.). register.microsoftpdc.com and sessions.microsoftpdc.com didn’t even live on the same server as the rest of the micorosoftpdc.com site.

This site used ASP.NET Webforms rather than ASP.NET MVC. We built the first release of the PDC2008 site in April - May 2008, and ASP.NET MVC was at a Preview 2 release stage back then. I love new and shiny, but I couldn’t in good conscience recommend that we launch the PDC site on MVC so early on. As I worked on it, though, I continually asked myself, “Will this be the last ASP.NET Webforms site I’ll deploy?”

Reuse Through Simplicity

We originally built a single site – microsoftpdc.com – but it’s turned into quite a bit more than that. The original microsoftpdc.com site was a pretty simple “brochure” site, designed to give people the information they needed to get to the PDC conference. The week before the conference, we shifted to a “live experience” mode that included a re-skin and changes to the sitemap to present information that was relevant to people attending the conference or keeping up with it online. That same week, we launched two other sites: m.microsoftpdc.com (a mobile site for PDC attendees) and 2009.visitmix.com, an information site for the MIX09 conference. While all these these sites looked pretty different, they all ran on the same code base. As I look back at things that went well, I think this is one thing we got right - we struck a good balance on the design so that we were able to heavily reuse the site code without having to tell the client that simple changes would take days because our system was too complicated.

Here are some of the things that helped keep us flexible:

• CSS based design
• Resisting the urge to premature generalization
• A great client – we were able to be involved in the user experience planning at a level which allowed us to define some consistency across the site
• Feed-based content as an ultra-lightweight CMS
• Master pages used on all pages in the site except the home page – for a while, the same site would display in “PDC Classic”, “PDC Live”, and “MIX” modes with a querystring switch (disabled in production)

The master-page / CSS switching worked surprisingly well – for instance, here’s the same agenda page shown before and after the theme switch:

We didn’t use the built-in theme system for ASP.NET, as it unfortunately doesn’t play that well with CSS based design and conditional stylesheets.

Look, Ma! No database!

One interesting thing about these sites is that they pulled their dynamic content from a variety of feeds. I built out a control which basically piped an RSS feed through an HTML cleaner to a ListView control. The ListView control just output the content in unstyled HTML, so it was reused all over the place. The HTML cleaner did a few things – stripped ugly HTML (including Word HTML markup) and did a “smart truncation” of the HTML content so we’d a desired character length with adjustments so we split at word boundaries and didn’t leave unclosed tags. I used the Html Agility Pack to handle the HTML parsing so I could focus on the logic.

The Mobile Site – m.microsoftpdc.com

While I set the technical direction and took care of some polish areas here, I really can’t take too much credit for what I consider to be one of the best features of the microsoftpdc.com site. David Shadle, the (awesome) creative director for the PDC online experience, designed a mobile experience exactly how I’ve always thought they should be done – as a focused mini-site specifically created for attendees at the conference. Then Jim Lin built it. I helped with some polish by following Scott Hanselman’s iPhone webdev tips (adding the iPhone home screen icon and adjusting the viewport), but mostly I just got out of the way and let it happen.

Here’s that fancy iPhone home screen icon. Phear my graphics skillz!!!

The Silverlight Live Experience

I put a lot of work into this, and it was probably one of my favorite features despite the fact that it was only live for 4 hours or so (during the webcast keynotes). We thought about how we watched conference keynotes – frantically switching between the video and Twitter – and we tried to set up an experience that would make people like us – geeks who love keynotes - really happy. We built a tiny Twitter client in Silverlight which pulled directly from search.twitter.com, and fortunately everything (Twitter included) held up well during the keynotes.

You know what took way more time than you’d expect? Styling the scrollbar in the Twitter area.

Hovering over the event photo thumbnails expanded to a Silverlight Deep Zoom photo explorer thing…

… and clicking on the thumbnails or spinning your mouse’s scroll-wheel would of course zoom you in on the photos, in all their Flickr medium-resolution glory! Of course, there were some dangers – if you zoomed in on the wrong photos, you could get stuck in a Deep Zoom recursion situation which has been known to collapse the very fabric of spacetime:

We leveraged a system Vertigo has  developed to enhance and manage Deep Zoom experiences. It’s called Big Picture, and it powers some other cool Deep Zoom projects like the Mojave Experiment site and this page which displays all the newspaper headlines with the news of the Obama’s recent election victory. In addition to handling UI details like laying out the photos in a nice justified rectangle, there’s a server piece running behind the scenes that builds the photo collections. We used the Big Picture server’s Flickr integration system, which ran a scheduled build process against the latest Flickr photos with the PDC2008 tag. That task took some management as the collection grew during the conference, but fortunately Jay Walters (the developer who did most of the work implemented the PDC / Deep Zoom experience) was on the ball, scaling our build times to match the photo collection size.

What did you think? What did we get right? What did we do wrong?

¹ I mentioned that I work at Vertigo, I love my job, and we’re hiring, didn’t I? Yes? Great!

Running Silverlight 2 on Google Chrome using the Chrome Dev Channel

When Google Chrome first came out and I read that it used Webkit, the same rendering engine that powers Safari, I tried browsing a few Silverlight 2 sites. It kind of worked, as long as the sites didn’t exclude browsers that weren’t on Microsoft’s official Silverlight support list. The controls loaded, but didn’t animate or update smoothly. While Microsoft still isn’t officially supporting Silverlight on Chrome, Chrome’s latest Dev Build (0.2.151.2) includes some specific fixes to support Silverlight 2 Beta 2. The information about the updates is in the release notes, specifically revision 1735:

The basic issue here was that the plugin would not paint correctly. The URLs mentioned in this bug load windowed silverlight plugin instances, which invoke the NPN_InvalidateRect API to paint. We send over the rects to the renderer, however these don't generate paints as the plugin is windowed. A peek at Safari's webkit implementation revealed that they merely invoke the InvalidateRect windows API in this context.

I followed the link in Jonas Follesø’s post over to the Chromium Developer instructions for running the Chrome developer build, which are really simple:

1. Download and run the Google Chrome Channel Chooser (http://chromium.googlecode.com/files/chromechannel-1.0.exe).
2. Click the circle next to Dev.
3. Click Update to save your choice.
4. Click Close.
5. In Google Chrome, click the wrench menu and choose About Google Chrome. Note: On Windows Vista, updates from the About box require Service Pack 1. With 0.2.149.29 (the current release), on demand updates do not work in Vista SP1 if User Account Control is disabled.
6. Click Update Now to install the current Dev channel release.
7. Restart Google Chrome.

And yes, it really is that easy – I had it running in under 5 minutes. First, I ran the Chrome Channel Switcher. It’s a simple EXE, not an install. Select Dev, press the Update button, then press the Close button:

Then open the about box (in the wrench menu) and click the Update Now button:

The update runs:

After the update finishes, close and reopen Chrome. If you want to verify you’ve updated, check the about box – you’ll see the version number is different:

That’s it. The whole process took just a few minutes for me. Now, off to view some of my favorite Silverlight 2 Sites! First, the Hard Rock Memorabilia page. There’s a check to see if the browser is supported, but it still lets me in if it’s not on the list (which is a good policy anyways, since it allows Linux / Moonlight users to view the site):

I hit okay, and we’re in. The site is snappy and everything works just fine:

How about the Mojave Experiment site, which uses a fancy Deep Zoom / video cloud thing? Yep.

Jonas has some screenshots of other Silverlight sites, which also worked well for him, and it worked for Samiq as well. From the release notes, this Dev release also fixes issues with Flash, such as YouTube videos freezing when you use the slider. Go get it!

Note: All the computers I tested this on had Silverlight installed before upgrading Google Chrome to the dev build. One had Safari / Windows installed, the other didn’t. Google Chrome doesn’t include Silverlight; you’ll need to install Silverlight separately if you don’t already have it.

Silverlight and Relative URI’s for Image and Video Sources

One of the main use cases of Silverlight is to enable rich media experiences, which requires referencing media files (images, audio, and video). That’s a little trickier than you’d expect, and it’s not very clearly documented. For instance, we ran into difficulty getting this set up in our Silverlight Advertising demo for MIX – we wanted to allow for a drag and drop experience to add a video as a canvas background without requiring additional downloading code to pull the video. Here’s an overview on how that works (thanks to Tim Heuer and for much of this information).

NOTE: This is especially important if you’re including a video or lots of images. The wrong way to include large media assets is to include them as resources (either compiled into the DLL or included in the XAP), since that will require your users to download everything before the Silverlight control is displayed. It’s important to keep your Silverlight XAP’s small so your users have a smooth load experience. For instance, the Hard Rock Memorabilia site browses gigabytes of images, but the XAP size weighs in at a tiny 69KB.

Silverlight URI References Are Relative To Your XAP

Short story is, Silverlight media Uri references don’t work the way you’d think, largely to keep compatible with WPF Uri’s which aren’t living inside a website. Relative Uri’s are relative to the Silverlight application, not the website. There’s no way to do site relative Uri’s without writing code. For example, using the following site structure:

Note: Screenshot has been updated since original post.

In this case, a source Uri could reference VideoB.wmv, but couldn’t reference VideoA.wmv or any images in /Images. So these are valid:

<MediaElement Source="/VideoB.wmv" />
<MediaElement Source="Subfolder/VideoC.wmv" />

These are not:

<Image Source="/Images/Logo.png" />
<MediaElement Source="/VideoA.wmv" />

Some Workarounds

1. Include the content in a subfolder of ClientBin. Set the content type to “Content” rather than “Resource” unless you want it downloaded with the XAP. Don’t include the media as a resource, or your users will have to download all the video and images before the Silverlight element is displayed.
2. You can manually move the XAP in the root of the site, and site relative paths will work. This solves the Uri issue, but then you’re on your own a bit because you’ve stepped outside of what Visual Studio’s tooling supports. Shawn Wildermuth recommends moving your XAP to the root of an application and has a good post on how to do it.

Pete Brown wrote a nice overview explaining how relative URI’s work in the three possible cases:

Images with Leading Slash (like <Image Source="/foo.jpg" />)

This type of references, with a leading slash, is root relative to the application root (the XAP). These files should by Content and have CopyToOutputDirectory set so that they are added into the XAP. If you inspect the Xap, you'll find the image in there.

When not found, the resource loading mechanism falls back to the application's site of origin and looks for the image under the same dir as the XAP is located. Note that this is the application's site, not the hosting page's site. That is an important distinction if you are creating cross-domain applications (where Site X has the page and Site Y has the XAP). If not found, an exception is thrown.

Images without Leading Slash (like <Image Source="foo.jpg" /> )

This type of reference, without any leading slash or anything, expect to find the image compiled into the assembly as a Resource. The path is relative to the path of the Xaml file from which they are being referenced. When not found, the ImageFailed event is raised

If you inspect the XAP, you will not see the image, because it will be in the assembly.

Absolute URLs (like <Image Source="absolute http url" />)

This will look at the named absolute Url as you would expect.

The absolute URL sounds like the simplest approach, and it is – from the Silverlight point of view. It’s not the simplest for web development, though, because your references won’t stay the same between your development, test, and production machines. In general, I use the leading slash references with the media files hosted at the “site of origin”, meaning the site at which the XAP file is hosted.

One More Gotcha / Workaround – WebClient and HttpWebRequest Don’t Follow The Above Rules

WebClient and HttpWebRequest use the browser’s network stack, so they work the way you’d expect for normal web applications. For example, in the following code:

var webClient = new System.Net.WebClient();
webClient.OpenReadCompleted += ProcessResult();
webClient.OpenReadAsync(new Uri("/info.zip"));

The web client request is routed through the browser’s networking stack, so it would be looking for info.zip regardless of where your XAP file is located in your site. I guess the downside there is that your location context changes depending on how you’re accessing resources (via Uri reference or by explicitly connecting to it via WebClient/HttpWebRequest). However, there’s an upside – your XAP can reference content anywhere in your site via WebClient/HttpWebRequest, even if the XAP is located in a subfolder. That’s handy, because in some cases that can be a lot simpler than determining the absolute URL of your resource.

MSDN Low Bandwidth Bookmarklet

There’s a semi-hidden feature in the MSDN Library website: Low Bandwidth view. We’ll talk about how to use it, why I like it, and some tips for switching it on and off. We’ll end up with an MSDN Low Band bookmarklet I whipped up to make it even easier.

The Low Bandwidth view has been available for a few months, but you wouldn’t know about it unless someone told you, since the only way to turn it on is to monkey with the URL. Try it - browse to:

http://msdn.microsoft.com/en-us/library/system.object.aspx

Now we’ll add the magic word: “(loband)” right before the “.aspx” at the end:

http://msdn.microsoft.com/en-us/library/system.object(loband).aspx

Magic!!!

Why LoBand is High Value

Simpler Layout

The obvious difference is that it replaces the navigation treeview on the left with a simple link. There are other subtle differences – simpler layout, fewer superfluous images. Higher signal to noise in my book.

Smaller Page Weight

The High Bandwidth version of this page weighs in at 100KB of HTML, but 400KB total by the time all the images, javascript, and CSS are loaded. Compare that with 66KB HTML / 70 KB total for the low bandwidth version. In this case (which is pretty representative) the low bandwidth version slimmed the page weight by 82.5%.

Faster Page Load

I’m not just talking about the smaller HTML here. The navigation tree on the left contains tons of nested unordered lists (ul>li>ul>li etc.). It’s actually a big improvement over the former HTML for that treeview, which (if I remember correctly) included a bunch of horrible nested tables with inline styles and javacript attributes. The new treeview uses a Telerik control, and outputs relatively clean HTML. Still, that treeview takes a while to load up – on my relatively quick development machine (with a very fast internet connection), the low band page loads twice as quickly – most of the time is spend in rendering the page. I’m not talking about milliseconds of difference here, I’m talking about 1 second load / draw time for low band vs. 4 second load / draw time for high band.

Who Cares?

Well, if you’re a Microsoft developer, you spend a lot of time on MSDN. There’s the time saver factor, sure, but more important is that fast load times removes the barrier to exploring the site.

Getting To The LoBand

The simplest way is to just add that (loband) bit before the .aspx file extension. If you’re on a page which already has one of those funky filters in the URL (like this: http://msdn.microsoft.com/en-us/library/cc189009(VS.95).aspx) you can just add a comma and put it in afterwards: http://msdn.microsoft.com/en-us/library/cc189009(VS.95,loband).aspx

When you switch to low bandwidth view by tweaking the URL, you  get a a “persist low bandwidth view” link at the top, which is nice. Clicking that link sets a cookie, so all MSDN you visit will be in low bandwidth view. That sounds great, but I find there are times where the low bandwidth is too low. Some pages (especially articles) are hard to read, and when learning a new object model the navigation tree is helpful. In that case, there’s a link at the top (where the “persist low bandwidth view” link used to be, before we clicked it) which unsets the cookie and returns us to the normal, high bandwidth view.

That all works, and I’ve used it since I heard about the low bandwidth view a few months ago. Still, it gets old – especially editing the URL the loband bit every time I’ve removed it.

Enter The Bookmarklet

Bookmarklets are great – they’re short Javascript functions that you bookmark, so you can run the Javascript on any page by opening the bookmark. They’re kind of like tiny Firefox addons. You can read more about bookmarklets, of course, on Wikipedia.

Well, after messing with that MSDN URL enough times, I figured it was time for a bookmarklet. So here it is: MSDN Low Band

It’s just a simple toggle – when you’re in the normal view, clicking the bookmarklet will switch you to Low Bandwidth view. Clicking it again will return you back to the normal view again. In Firefox / Opera / Safari, you can just drag that peachy colored button to your Links toolbar. In Internet Explorer, you’ll need to right click the link and select “Add To Favorites”, making sure to save to the Links favorite folder. I’ve tested it in IE, Firefox, and Safari.

The Delicious.com Bookmarklet installation help page has some nice screenshots and more information on installing Bookmarklets. If you’re interested in writing your own bookmarklets, I recommend you use a web based Bookmarklet helper page to simplify the grunt work, like this one.

So How Does It Work?

It’s really simple. At first I messed with the URL, but then I figured out that it was simpler to just modify the cookie and reload the page. Here’s the code, formatted so it’s easier to read:

javascript:
if(document.cookie.indexOf('LoBandEnabled=yes')<0){
  document.cookie='LoBandEnabled=yes;path=/;domain=.microsoft.com;%20expires=Wed,%2001-Aug-2040%2008:00:00%20GMT';
}
else{
  document.cookie='LoBandEnabled=no;path=/;domain=.microsoft.com;%20expires=Wed,%2001-Aug-2040%2008:00:00%20GMT';
}
window.location.reload();

Subtext 2: OpenID Login Support

The recent Subtext 2 release includes a feature I worked on: OpenID login support. Let’s take a quick look at how you use it, then we’ll talk about the how the code works and why it’s a useful feature.

What’s OpenID?

I’m a big fan of digital identity, and could ramble on about OpenID for a while. But if I did that, this blog post would be published sometime after Windows 7 ships. Fortunately I pushed OpenID on some people who are a bit more prolific than I’ve been of late, so I’ll refer you to this post by Jeff Atwood (CodingHorror) and this webcast by Rob Conery. Scott Hanselman wrote a great overview of OpenID, as well.

The elevator pitch: Rather than being issued an account at every single website you login to, you issue them one. You establish an OpenID URL, which only you can login to, and then you give it as your account information to sites which support OpenID authentication. You’re in control of your account, you’ve got a central place to manage your password, etc.

OpenID is a generic account that you can reuse on other websites.

Too Hard! Couldn’t I Just Fax Someone My Birth Certificate?

If you haven’t used OpenID, this is going to look complex. That’s because I’m showing you the hard case – it’s like learning to play Stairway To Heaven when you don’t have a guitar or amp. The first time through, we’re going to have to run down to the pawn shop to get you a Fender knockoff and a thrasher amp, but the next time you’re ready to rock you’ll be all set. So hang with me while we get setup, and at the end I’ll show you how it will work when you log in to your site tomorrow.

Step 1: Get an OpenID

There’s a good chance you’ve already got an OpenID, since many popular services like Flickr, Yahoo, and Blogger are OpenID providers, meaning that you can user your account with those services as an OpenID identity:

If you don’t have an account with any of those services (or these on the OpenID public providers list), I recommend signing up with MyOpenID.com – they’ve got great support, and rich security features if you want to use them, such as InfoCard integration and phone verification.

Step 2: Tell Subtext What OpenID URL You’ll Be Using

The Security / Options tab has a new location where you can enter an OpenID URL. It’s important that you get this right – we’ll try to clean this up for you, but I recommend you type this as exactly as possible. In my case, my OpenID URL is http://jongalloway.myopenid.com/, not jongalloway.myopenid.com. Even the trailing slash is important.

Note that to in order to make this setting, I’ve logged in to Subtext using my standard Subtext username and password. That login doesn’t go away when I setup OpenID authentication, I’ve just enabled an additional security feature – I’ve got two ways to login to my blog now.

Step 3: Login Using OpenID

Now that Subtext knows my OpenID URL, I can use it to login to the site. The login dialog includes an OpenID sign-in prompt at the bottom, so I enter my OpenID URL in the prompt and click Login.

Now here’s the part you may not be expecting if you haven’t used OpenID – I need to login at my OpenID provider. That’s not such a big deal, though, because myOpenID (and many other providers) have a “Stay Signed In” option option, which is appropriate if you’re logging in from a computer which is in a secure location (your home, a work computer you keep locked, etc.). The Versisign Seatbelt Firefox Extension is a pretty handy way to sign in to your OpenID identity once for a browser session, too.

Now, the first time I log in to myOpenID from my Subtext blog, myOpenID is going to ask me if that’s cool. Again, a little unexpected if you haven’t used OpenID before, but this is a one time thing. We’re telling myOpenID that my Subtext blog is going

So, I type in my password and click the Sign Button, and my OpenID provider redirects me back to my Subtext instance (with an “authenticated” message), and Subtext logs me in:

Fine. Now Show Me Easy.

Thanks for hanging in there. Here’s how it looks tomorrow, providing you’ve checked that “Stay Signed In” checkbox. First, we browse to the login screen, enter our OpenID URL, and click Login:

Now, we’re automatically logged in and brought to our admin screen:

If you hadn’t checked that Stay Signed On checkbox, you’d get one screen in the middle – the login page for your OpenID provider’s page.

How’s It Work?

That’s the subject of another post, but let me show you one quick screenshot which shows the HTTP traffic during that last login.

That shows the general sequence of events:

1. I requested the Login page (it’s running on my local machine – 127.0.0.1)
2. The DotNetOpenID login control makes a request to the URL I provided, saying “Yo. My URL is 127.0.0.1:2732, can you authenticate the user and verify that I’m on their list of sites?”
3. There’s a little negotiation between the two sites, after which myOpenID returns an “Okay” message via SSL.

Show Us Your Code

Sure. For this release, I just used the DotNetOpenID.OpenIdLogin control, which is as simple as dropping the control on the page and handling the LoggedIn event. Scott Hanselman wrote about this before when he set up OpenID on DasBlog. In the case of Subtext, here’s the code I added for that LoggedIn event:

protected void btnOpenIdLogin_LoggedIn(object sender, OpenIdEventArgs e)
 {
   e.Cancel = true; //required to prevent logging everyone in
   if (e.Response.Status == AuthenticationStatus.Authenticated &&
       SecurityHelper.Authenticate(e.ClaimedIdentifier, chkRememberMe.Checked))
    {
         ReturnToUrl(Config.CurrentBlog.AdminHomeVirtualUrl);
    }
} 

You can see it in context in the Subtext SVN browser. Now that I’ve worked with it, I’d like to ditch the OpenIdLogin control for a future release. It works just fine, but it generates HTML that I’m not very happy with (table based markup, not CSS friendly). In the future, I’d probably write my own control and just use the libraries which are included in DotNetOpenID – they’ve been great.

OpenID Passthrough

There’s another new OpenID feature in Subtext 2.0 – OpenID Passthrough. The idea there is that you can use your blog URL as your OpenID URL, and it just redirects over to your “real” OpenID provider. Let’s assume that my blog was deployed to http://jongalloway.com; in that case I could make the following OpenID Passthrough settings on the Subtext / Configure screen, after which I could use http://jongalloway.com as my OpenID URL.

Upgrading to Visual Studio 2008 / .NET 3.5 SP1

Have your Visual Studio Installation Media Handy

I was prompted for the original installation media both when uninstalling the SP1 Beta and when installing the SP1 RTM. The file it’s looking for is vs_setup.msi in most cases, but they’re not all created equal. If you installed from a Visual Studio 2008 Professional DVD or image, you can’t just insert a Visual Studio 2008 Standard DVD. In my case, I installed of in ISO image, so I mounted the iso file (en_visual_studio_team_system_2008_team_suite_x86_x64wow_dvd_X14-26461.iso) as my F:\ drive and browsed to vs_setup.msi.

You don’t need it yet, but you don’t want to get half way through the upgrade and realize your install media is back at home, or that you deleted the ISO file and you have to wait for the 4GB download. Got the install media? Great, onward!

Use the Visual Studio 2008 Service Pack Preparation Tool

I think the biggest tip here is to use the Visual Studio 2008 Service Pack Preparation Tool. The original help information SP1 Beta (since updated) had a long, involved un-installation process, so people who’d had to uninstall that before figured we’d have to go through that process again. I had to a month ago, and it didn’t go very smoothly for me – probably due to some internal, pre-release builds of Silverlight 2 I’d installed at the beginning of the year. I ended up having to uninstall everything developer related to fix a Silverlight Package Load Failure error.

So – that’s all been simplified now, and in most cases it sounds like it’s working great. I ran it on two of my computers (which have been littered with alpha and beta stuff) and the Visual Studio 2008 SP1 install ran flawlessly. I’ve been listening on Twitter, and it’s been a smooth upgrade for most people (notable exceptions: Rick Strahl, Sam Gentile).

Heath Stewart's post on the Service Pack Preparation tool indicates that the SP1 install will block says the SP1 install if you've installed the SP1 beta at any point, so the main reason I'm calling the Service Pack Preparation Tool out here is to save you from going through an unnecessary manual install only to get the prompt indicating that you still need to run the Service Pack Preparation Tool.

According to Heath:

The tool will verify Visual Studio integrity and remove previous Visual Studio 2008 updates or pre-release software

• Microsoft Visual Studio 2008 - KB945140 (Beta)
• Microsoft Visual Studio 2008 - KB944899
• Microsoft Silverlight Tools Beta 1
• Microsoft Visual Studio 2008 - KB949325

So if you've never installed any of those betas you don't need to run the prep tool, however based on my experience installing Visual Studio over the years, I like the sound of "verifying Visual Studio integrity" before installing. I'd expect that the tool would run really quickly if you haven't installed the beta.

So, I'd recommend that you grab the Service Pack Prep Tool and let ‘er rip!

Allow some time for the Service Pack Prep Tool - it took 45 minutes on one machine, and others have reported that it ran for 8 hours or more. For those reasons, you might want to take some precautions to make sure you don't have a prompt holding up your install if you're going to be away from your computer while it's running. Here are a few tips there:

1. As I mentioned before, you should expect to see a prompt for the installation media during the uninstall. If you can insert the media into the same drive you installed from in the first place (easy if you installed off a DVD drive, not so easy if you just attached an ISO file), you won't have to watch your computer for the media installation prompt.
2. There's a check for process which will require a reboot when you're done. If you're going to be away from the computer while, I'd make sure to shut down your browsers and the Vista Sidebar (not sure why, maybe uses managed code).

Running the Visual Studio 2008 SP1 Installer

There are four installs listed on the Visual Studio 2008 Service Pack 1 (SP1) and .NET Framework 3.5 SP1 Downloads page:

• Visual Studio 2008 SP1
• .NET Framework 3.5 SP1
• Visual Studio 2008 Express Editions with SP1
• Visual Studio Team System 2008 Team Foundation Server SP1

The Visual Studio 2008 SP1 install includes .NET 3.5 SP1, so that'll do it. Again, be ready for that installation media prompt and the incompatible processes check.

Reinstalling The Silverlight Tools

The Service Pack Prep Tool removes the Silverlight Tools (as it should) so you need to reinstall those when the SP install completes. They've just updated the Silverlight Tools installer (silverlight_chainer.exe) in-place, so you can grab it from the same location it's always been at: http://www.microsoft.com/downloads/details.aspx?FamilyId=50A9EC01-267B-4521-B7D7-C0DBA8866434. Tim Heuer's got more information on Visual Studio 2008 SP1 and Silverlight Tools.

Great - So What Did I Just Install?

I think some of the most interesting features are ASP.NET Dynamic Data, Entity Framework, ADO.NET Data Services (REST based), XBAP and ClickOnce support for Firefox, the Client Profile (24MB lightweight .NET framework which is makes it easier to distribute .NET applications to users who don't have the .NET framework installed).

There's a lot more, though. Rather than writing up "yet another SP1 feature list" I'll point you to a few which I've found helpful:

First off, you didn't install ASP.NET MVC, although you got a building block (System.Web.Routing - similar to URL Rewriting, but it's bi-directional). Phil Haack explains it all in a post titled (oddly enough) ASP.NET MVC Is Not Part of ASP.NET 3.5 SP1.

ScottGu's out of the office right now; fortunately Scott Hanselman stepped up with good overview of what's in SP1.

Here's what changed from a performance point of view.

Dr. Tim Sneath wrote a great post on the WPF enhancements in SP1 from when the Beta was released.

Here's the overview at MSDN: Visual Studio 2008 Service Pack 1 and .NET Framework 3.5 Service Pack 1

If you want the point by point list, take a look at the following KB articles:

• 950263 List of changes and fixed issues in Visual Studio 2008 Service Pack 1
• 951845 List of changes and fixed issues in Visual Studio 2008 Service Pack 1 for Team Editions
• 950264 List of changes and fixed issues in Visual Studio 2008 Service Pack 1 for Express Editions
• 951847 List of changes and fixed issues in Visual Studio 2008 Service Pack 1 for the .NET Framework 3.5

And if you really want the low level details, Patrick Smacchia used NDepend to show exactly which classes changed in SP1.