Tales from the Evil Empire

Bertrand Le Roy's blog


Bertrand Le Roy

BoudinFatal's Gamercard

Tales from the Evil Empire - Blogged

Blogs I read

My other stuff


September 2007 - Posts

JSON-style dictionary parameters in C#?

Eilon has an interesting post about using the new anonymous object initializer syntax in APIs that take dictionaries. The end result looks very much like the JSON parameterized function calls that are very common in many JavaScript frameworks, which shows once more how C# is getting many of the nice features of dynamic languages while remaining statically-typed. While I wouldn't use that just to get named parameters like those frameworks are often doing, it looks well-justified in this context because the parameter it's being used for is actually not predetermined.


OpenAjax InteropFest 1.0: Microsoft's entry

Yesterday I got to write our entry in OpenAjax's InteropFest. The goal of this event is to demonstrate how different Ajax libraries can be parts of the OpenAjax ecosystem and interact with each other through the OpenAjax hub. The currently central feature of the hub is to expose a publish/subscribe message bus so that both producers and consumers of events can speak through a third party that is neutral to specific Ajax implementations.

The OpenAjax Alliance provides a template for demo applications that shows a live data source feeding fake stock quotes through the hub to a visual component that then renders them.

I've chosen to modify the live data source to be a Microsoft Ajax-style component. The visual component would not have been as interesting because it's really the application that is a consumer of events. The visual component itself doesn't strictly speaking communicate with the hub, the application does (through a trivial API call) and then transmits the results to the component for rendering. In other words, the data source is a lot more interesting because it exposes the event, and if it is a Microsoft Ajax component, it will use the Microsoft Ajax event pattern, which is quite different from the OpenAjax event pattern.

I did refactor the consuming part of the application quite heavily, but that was more to understand it than anything and it's not the essential part of our entry.

To achieve the mapping from Microsoft Ajax events to OpenAjax messages, I've written the following simple helper:


Sys.OpenAjax._Helper = function() {
Sys.OpenAjax._Helper.prototype = {
function(owner, eventName, messageName, publisherDataMapper) {
var handler = function(sender, args) { OpenAjax.hub.publish(
publisherDataMapper ?
publisherDataMapper(sender, args) : null); } owner["add_" + eventName](handler); return handler; }, unmapEvent:
function(owner, eventName, map) { owner["remove_" + eventName](map); } } Sys.OpenAjax._Helper.registerClass("Sys.OpenAjax._Helper"); Sys.OpenAjax.Helper = new Sys.OpenAjax._Helper();

The application code can use that new API to map the events of any Microsoft Ajax component to become publishers of OpenAjax messages without any necessity for the component to contain any code specific to OpenAjax, let alone know about it. In other words, the component implementation remains completely decoupled from OpenAjax and it's the application that makes the connection.

The application maps the quoteChanged event to the org.openajax.interopfest10.datagen.stockpriceupdate OpenAjax message like this:

    corpList, "quoteChanged",
    function (sender, args) {
        var quote = args.get_quote();
        return {
            tickerName: quote.symbol,
            corpName: quote.name,
            price: quote.price

From this moment on, all quoteChanged events will be published on the hub. An interesting thing to note is the use of an optional function that maps the Microsoft Ajax event argument to the expected OpenAjax data payload.

Brad Abrams kindly hosts the demo on his personal website for the moment:

The source code can be downloaded by clicking here.

Should I use a type attribute on script tags?

I have immense respect for Douglas Crockford, but that doesn't mean I can't disagree with him... Douglas has written on a number of occasions that he prefers plain script tags, with no attributes. The argument goes like this:

"This script block uses the language attribute. This was a feature that was introduced by Microsoft in order to support VBScript. Netscape then adopted it to support its own nonstandard deviations. W3C did not adopt the language attribute, favoring instead a type attribute which takes a MIME type. Unfortunately, the MIME type was not standardized, so it is sometimes "text/javascript" or "application/ecmascript" or something else. Fortunately, all browsers will always choose JavaScript as the default programming language, so it is always best to simply write <script>. It is smallest, and it works on the most browsers."

What he forgets to mention here is that if you care about XHTML compliance, you can't make the economy of the type attribute. I also think it's good practice as we don't know if in the future browsers won't support other scripting languages, even if the default will probably always remain JavaScript (and even if browsers don't support them, we've used our own type in the past for xml-script and handled the parsing ourselves). It's also more expressive.

The second thing that I think is not quite true here is that the MIME type wasn't standardized. MIME types are standardized by IANA (Internet Assigned Numbers Authority). IANA lists both "text/javascript" and "text/ecmascript" as obsolete, pointing to RFC 4329. The valid types are "application/javascript" and "application/ecmascript". All modern JavaScript interpreters being implementations of ECMAScript, it seems like it would be best to use "application/ecmascript".

Except that our good friend IE doesn't recognize them and ignores the script altogether if you use them. So once again, thanks to one more non-standard behavior of IE, we can't afford to follow the standard.

So to sum this up, if you don't care about XHTML compliance or expressiveness of your markup, it's probably safe to use a plain script tag. If you do, you should use "text/javascript" until IE gets its act together (if that ever happens).

Plastic X-plorer paint job

The guitar Harmonix chose for the Xbox 360 version of Guitar Hero II is not exactly my favorite guitar. The Gibson X-plorer just reminds me too much of german hard-rockers from the eighties... and mullets. Harmonix' plastic rendition of the guitar doesn't look any better than the original of course, and white doesn't help. I've seen things at Toys'R'Us that look less toy-like. I can't do much about the shape but seeing that I had already done a red paint job on my faceplate a while ago, I thought that at least the color had to change.

Here's what it looks like now:

Red Xplorer guitar

So how do you make one of these then? Well, here's how. First, here's what you'll need:

What you'll need

  1. A respirator. Don't even think about doing that job without one. That paint is really dangerous stuff, don't mess with it. A respirator will cost you less than replacement lungs, and it will make the spraying comfortable, with no nasty odor or aftertaste... Don't use a dust mask, those are for dust, not chemicals. Use a proper respirator. Also, do that in a well-ventilated place. Follow the safety instructions on the paint cans.
  2. Not just any plastic paint. What you need is this kind, which gets into the plastic and changes its color. Instead of adding a layer of color, it chemically reacts with the plastic and colors it in the mass. Once it's dry, it has the same texture as the original surface, is not sticky and doesn't peel. But it's really toxic, so again, you need #1. The paint is easily found at arts and craft shops such as Jo-Ann.
  3. Latex gloves. And working clothes. You'll get paint on them, so don't wear your best suit for that.
  4. A small Phillips screwdriver is enough for the whole project.
  5. A bunch of brushes if you want to add your personal touch, for paint and varnish. Not necessary if you just want to change the color. For the motto, which is really precise work given how small it is, no brush was fine and precise enough, so I used a small nib.
  6. Some acrylic paint for the personal touch.
  7. Varnish ditto.

And of course, a guitar. Be warned that this is work you should do very carefully. If you screw up, you may well not have a usable or presentable guitar in the end, and your warranty will be voided, so do that at your own risk. Also be warned that the paint takes almost a week to be completely dry so you must be prepared not to play Guitar Hero in the meantime.

The first step is to take apart the guitar completely. This is not very difficult, it's putting it all back together that's more challenging if you don't remember what goes where. So the thing you need to think hard about as you're dismantling the axe is what goes where (especially screws). Take notes and draw schemas if necessary, put all the little screws in separate boxes and try to remember what type of screw goes where. The pictures and instructions below should help.

There are five types of screws in the guitar:

Five types of screws

The black ones are for the case, the silver ones for the inside (circuit boards, triangular plate, etc.). The only thing you don't need to remove is the circuit board that keeps the fret buttons in place. All the rest must be removed (there are just four screws on the strum bar's circuit board -those in the middle as seen on one of the photos below- which should stay in place), after which you should have the three parts to paint (two halves of the case and the triangular face plate):

What to paint

And a mess of cables, circuit boards and plastic pieces like this:

The insides of the guitar

So now it's time to spray the paint on the three white parts of the case.

Start by cleaning the parts carefully. You don't want any dust or hair or anything on the surface, before or after you apply the paint.

Take your time to apply the paint. It takes three layers at least to get a uniform color so make quick, light passes, don't overpaint as you can't sand that stuff afterwards (it's in the mass, not over the plastic). After the first pass, you'll see plenty of white, which is fine. Take care to hold the piece for a while before you set it down, and be super careful while doing so as not to hit or rub against anything. What I've done was to hold the piece from one of the numerous screw tubes on the inside surface (which remains unpainted) and put it down on a table with only the part from which I'm holding not on the table's surface. The idea is to be able to put the part down for drying while not touching the painted surface. When you've done enough passes to have a uniform color, take care of any remaining touch-ups, such as the seams: make sure that no white will still be visible there once you put the guitar back together. You must also know that while the chemical reaction is taking place and until it has fully dried, the plastic will be relatively soft and more vulnerable to any kind of impact or even to your fingernails. The surface will remain a little sticky to the touch for about a week (in my experience) so keep the guitar in a safe place during that time. The stickiness was actually what I used to determine when it was dry. The surface of the guitar feels now exactly like before but it was not the case during that week after painting. Still, it's relatively safe to reassemble the guitar after an hour or so if you take care of avoiding hitting anything.

Here are the parts after the paint has been applied:

The parts after painting

Check out how even the smallest details are still exactly the same, there is no additional layer on top of it, the color is really in the mass:

All details are preserved.

Once everything dried reasonably, it's time to reassemble. Start with the triangular face plate and then move on to the control panel. The three buttons are very easy, but the d-pad requires a little more attention. screw the front and back parts back together, then put the rubbery contact sheet in place:

Control panel reassembling

Once this is done, you can screw back the main circuit board over the control panel buttons:

Main circuit reassembled

It's now time to put the strum bar back in its hole, and then the two plastic parts that will maintain it in place. Don't put any screws yet.

Reassembling the strum bar.

The circuit board will maintain all three parts once the four screws are put back. Check that the strum bar moves normally.

The strum bar circuit reassembled.

The whammy bar can then slide in its place. There's nothing to maintain it in place, it's the case itself that will secure it.

Reassembling the whammy bar.

Secure the extension connector (the one that looks like a network or phone jack).

Slide the headphone connector's small circuit board into the black plastic casing.

Reassembling the headset connector.

Just leave some room to be able to secure both parts against the case and both screw holes.

Headset connector done.

Secure the black wire with the small plastic part with two holes, secure the connector in the case, and then put the neck back and attach it to the case with four screws. Make sure that no wire is over a hole (they could get damaged when you close the case, and as a matter of facts one of the wires on the photo below will have to be moved slightly to the left of the whammy bar). Just inspect the whole thing, it should look something like this:

Done with the inside.

You can now close the neck and secure it closed:

Closing the neck.

It's time to close the case. Don't forget the strap's button that gets secured between the top and bottom of the case. Note that the photo below makes it pretty clear where you need screws.

Time to put the final screws back.

And we're almost done...

Time to apply the personal touch.

All that remains to be applied is the personal touch. Choose a place out of the way, to prevent your Guitar Hero routine from wearing out your work over time (if I redid this, I would write slightly to the right to get it out of the way). I used acrylic paint over the triangular faceplate and then varnish over it. If you wonder about the rougher look of this part of the guitar, I have to admit I screwed up here. I accidentally dropped the part right after painting it. So I just chose to roughen it up and give it this look using a soft cloth Dremel tool and redo it. In the end, I quite like it.

Ceci n'est pas une guitare. Je ne suis pas un héros.

I chose to write "Ceci n'est pas une guitare. Je ne suis pas un héros" in an obvious reference to Magritte's Treachery of Images and of course to make it super-clear that yes, I know this is not a real guitar. It means "This is not a guitar. I'm not a hero."

As a final note I'd like to acknowledge that the inspiration for this work came from Alcaron's Noir 360 project (a black controller that he did way before the Elite).

Posted: Sep 09 2007, 10:30 PM by Bertrand Le Roy | with 8 comment(s)
Filed under:
How to build a cross-browser history management system

When we built the history management feature in ASP.NET Futures, we spent considerable time experimenting with the different behaviors of the main browsers out there. The problem with such a feature is that it has to rely on a number of hacks because browser vendors basically never anticipated this need. Now they're thinking about it, so all this may be simplified in a few years, but in the meantime, it's a very complicated feature to build. One of the things that struck me was how little reliable literature is available on the subject. There is a lot of partial information, lots of false or unverified information, but very little that's really comprehensive, reliable and up to date. Good references I found include Brad Neuberg's Really Simple History and Handling Bookmarks and Back Button as well as Mike Stenhouse's Fixing the Back Button and Enabling Bookmarking for Ajax Applications. But it was a lot easier to just experiment directly on the different browsers and verify our theories directly.

This blog post will attempt to be an account of the difficulties we encountered and how we worked around them. It's also a brain dump that will help me and others maintain the feature in the long term. I hope it will evolve over time as feedback comes in, browser bugs get fixed and we ourselves learn more.

Note: all code in the pages attached to this post is pure JavaScript and XHTML and has no dependency to any script library.

How history used to work

It used to be the case that history in the browser just worked: the user navigated from page to page following links and the history stack just got filled, the back button worked as usual and life was good. With dynamic pages that post forms, things got a little less ideal as every post action would make a new entry, no matter if or how little the state of the page changed. Plus, you got this nasty "do you really want to repost this form" dialog that most users had no clue what to do with.

But Ajax applications broke the model in a more fundamental way because most actions are done without navigating away from the page or posting forms. This means that the browser basically has no way of knowing the state of the application changed in a meaningful way and nothing new gets in the history stack. Any naive user who doesn't know the implementation details of your application will expect the back button to work like it has for decades, which is catastrophic because by navigating back to the previous page it knows about, the browser will lose all the user's precious work.

Thanks to a few hacks such as the ones I will expose here, it doesn't have to be that way, and the control of the history stack can be given back to the application developer. We can even make the model better than it used to be and choose exactly what constitutes a meaningful enough state change to warrant an entry in history.

Navigating without moving

The main trick that history managers use is to have the browser believe the user navigated to a new url without the current page and all its JavaScript and DOM state being thrown away. The only part of the url that enables such a thing is the hash part. The hash part is what comes at the end of the url after a pound (#) sign. The original intent of this part of the url was to allow for navigation inside of the document. You would put a special named, href-less anchor tag in your document, and then navigating to #nameOfTheAnchor would just scroll the anchor into view. The page doesn't get reloaded, but it does enter the browser history (almost, we'll see about that in a minute). This is great for long pages with a table of contents for example.

But it's also great in that it's a way for us to manipulate the history stack without unloading the page, which is exactly what we need. So this is one more example of using something for a totally different use than what it's been designed for, but what choice do we have? And it's actually all right as this new usage doesn't conflict with the old one.

So it seems like we have an easy way to get this to work: whenever you want to create a history point, just navigate the page to #SerializedFormOfTheStateOfTheApplication; detect url changes somehow (polling works (almost) everywhere) and re-hydrate the application state from that. That should work, right? Yes, it should, but it doesn't, except on Firefox and Safari 3 for the Mac, for various reasons.

I've prepared a page that implements exactly that logic that you can try here:
HistoryHash.htm.txt (download and rename HistoryHash.htm to run on your system)

Here's a summary of the results on some popular browsers:


IE 7

Firefox 2

Opera 9

Safari 2 Mac

Safari 3 Windows beta

Safari 3 Mac beta
URL changes Yes Yes Yes Yes Yes Yes
Creates history entry No Yes Yes Yes Not on first page Yes
Back doesn't kill timers Yes Yes Not in 9.23 and 9.24
Yes in 9.10
Yes Yes Yes
Back enables forward Yes Yes Yes Yes Sometimes Yes
location.hash reflects changes in the url Yes Yes Yes No Yes Yes
Making it work in IE

The problem in IE comes from the fact that it doesn't create a history entry when the hash changes. This can be worked around by navigating an iframe in addition to changing the hash as IE does create a new history point every time a frame is navigated. The problem with that technique is that just like the main page, the iframe can't be just navigated to a new hash, it needs to move to a different page, which means a round-trip to the server on every new state, which is quite wasteful. I've heard of techniques to create the iframe's contents through clever scripting in the iframe url and document.writing, but I've never been able to consistently reproduce it.

<update date="Sep. 14 2007">
One of the comments over at Ajaxian pointed me to this great article from Julien Lecomte, where he explains how they built the feature over at YahooUI. It's a super-interesting read and thanks to it, the document.writing trick suddenly clicked together for me. The trick is that you need to open the document before you write to it and close it when you're done if you want the history point to be created by IE. Somehow I had missed that but today I tried again and had no difficulty making it work. The only thing that may be a little tricky is that you need to pass a url into document.open or the browser will default to about:blank. That may look ok until you try it under https, in which case you're going to get the infamous dialog about mixed contents. This can be solved by using the usual magical url javascript:'<html></html>' (thanks to the toolkit team for that trick). Please note that now we don't even have a single request for the frame, even on the first hit, again thanks to the magical url. It's also important to note that the iframe must absolutely be in the static markup of the page (you can't create it dynamically) and on the first request it must point to an existing page on the server.
I've updated the sample page for IE below to include this trick and I'll also integrate it into ASP.NET Ajax.

The frame gets the state passed to it through the search part of the url (the part after a question mark). It then calls back into a function in the main document when it loads. When the user navigates back to a previous state, it's the iframe that really navigates back, not the main page. When it does, its load event fires, which calls back into the main page again. From that callback function, we set the hash so that the browser url reflects the state for bookmarkability, and of course we update the application with that state.

Here's a simplified implementation of this. I didn't hide the iframe to make clearer what is happening here but that's of course what you'd do in production code.
HistoryIE.htm.txt (download and rename HistoryIE.htm to run on your system)
You will also need the iframe file:
HistoryFrame.htm.txt (download and rename HistoryFrame.htm to run on your system)

<update date="Mar. 31 2008">
Internet Explorer 8 finally gets rid of the need for the iframe navigation: changing the url fragment now results in a new entry being created in browser history. We were able to make our implementation of history work in IE8 by just restricting the iframe code to versions earlier than 8. Internet Explorer now follows exactly the same code path as Firefox, Safari 3 and Opera 9.5. Not all is perfect though as IE still has this weird quirk where when navigating away from the history-managed page causes its history entries to collapse to just one entry. If you do go back, it will expand again but this is really confusing to the users.
On the bright side, IE is the only browser as I write that implements an HTML5 event that gets triggered when the url fragment changes, eliminating the need to poll. I really hope the other browser follow that lead...

Making it work in Safari 2

While the workaround for IE results in a completely different implementation, at least it makes some sense and is consistent. With Safari, we're lost in bugland. In Safari 2, monitoring the hash from a timer doesn't work as location.hash is not getting updated as the url in the browser's address bar changes. This is a plain bug that is fixed in Safari 3 and the current WebKit builds. The only information we have that gets correctly updated when the user navigates through history is history.length. In other words, we know the index of the history point but we don't know the corresponding state. One thing we can do to work around this is to maintain our own array of states and use the index to retrieve the right state from this array. One caveat is that if we use a JavaScript variable to store this array, navigating to a different page will completely wipe out the state array and kill the history feature. This can be partially worked around by storing the information into a hidden form field instead of a JavaScript variable: form field values are restored by the browser when navigating through history. This works relatively well to maintain history even if the user navigates away to a different page and comes back but what it doesn't handle on the other hand is the user pressing F5.

Here's an implementation of those workarounds (I've left the history stack field visible to make it easier to understand what happens but of course in a real application it would be idden):
HistorySafari2.htm.txt (download and rename HistorySafari2.htm to run on your system)

This must be fixed in Safari 3, right?

Well, yes, Safari 3 Mac does the right thing as the unmodified hash code that works on Firefox just works. But on Windows currently other bugs cripple the feature. In the current nightly build and public beta of Safari 3 for Windows, if your page is the first that you load in the browser, no history entries get created when the hash is modified. Another weird bug that I couldn't quite find reliable repro steps for is that under some circumstances, hitting back does not enable forward, making time travel only possible to the past, without any hope of return. Somebody give them a flux capacitor, an old clock and a thunderstorm... Because of this, the Safari 3 for Windows implementation kind of works sometimes but it really needs to be fixed. I filed bug 14080 in WebKit a while ago to track this. Seriously, I don't blame them, it's a beta, betas have bugs and I'm pretty confident this will get fixed pretty soon. I've had pretty good experience with their reactivity in the past.

What about Opera then?

Well, it used to work perfectly well in Opera up until version 9.10. Then they broke it. In 9.23, and also in 9.24 which is the current version as I'm writing this (not a beta), for some reason Opera cancels all timers when hitting back. This completely breaks any history manager because there is now no way of knowing the user navigated. This problem has been reported by several persons on the Opera forums and apparently there's a bug open for it, but the closedness of Opera's bug database doesn't allow us to monitor the status of the issue.

<update date="Sep. 17 2007">
Neil Jenkins pointed me to a nice trick he's come up with to work around this bug. The idea is to have a hidden image with the following src attribute: javascript:location.href='javascript:onTick();';
Amazingly, this works and the code gets executed when the user hits back. Just doing javascript:onTick(); directly doesn't work for some reason that I don't quite understand. Apparently, Opera tries to be smart about what can be run as an img src, but it's easily worked around. If you ask me, script should simply not be allowed as image urls (as well as in a number of places) but I suppose that would break too many hacksapplications.
Anyway, the workaround does work, and it's simple enough that you should use it if you need to support Opera 9.23 and 9.24. But you also need to know that this is fixed in Opera 9.5 (still in beta as I'm writing this) so it may not be worth fixing in the long run. Kudos to Opera for their reactivity on this issue.
HistoryOpera.htm.txt (download and rename HistoryOpera.htm to run on your system).

The state of things

So things are in a pretty grim state currently. It seems like we're going back (pun intended). We used to have a collection of tricks that made possible an implementation of a history manager that worked pretty well in IE, Firefox, Opera and Safari. Now, we only have IE, Firefox and Safari Mac. I just hope this is only temporary and that both Apple and Opera repair their browsers soon.

<update date="Sep. 17 2007">
As explained in the previous section, there's now a known workaround for the Opera bug, and the bug itself will be fixed in 9.5.

<update date="Feb. 18 2008">
The WebKit bug seems to be fixed.

<update date="Apr. 18 2008">
Apparently I forgot to mention one annoying bug in Firefox that url-decodes the hash automatically, which gets in the way of using the hash as a querystring-like state bag. More details in the bug comments here: https://bugzilla.mozilla.org/show_bug.cgi?id=378962#c4.

<update date="Dec. 22 2008">
Some great additional information on ways to help refresh behave better on IE can be found here: http://www.overset.com/2008/12/21/internet-explorer-iframe-browser-history-lost-on-reload/

What's next?

In the next posts, I'll get into more details about the initial request, maintaining meaningful titles and mixing all this with asynchronous operations, which is where it really gets tricky.

More Posts