Tales from the Evil Empire : JSON

Microsoft Ajax 4.0 Preview 4 now available

Bertrand Le Roy — Wed, 18 Mar 2009 07:49:00 GMT

The Microsoft Ajax team made the fourth preview of the 4.0 version available on CodePlex. This is an important release because it enables the full client data story, complete with the ability to get changes back to the server automatically.

Here’s a quick recap of some of the available features:

Getting a client representation of data from an ADO.NET and REST data service.
Rendering data on the client using templates.
Declarative instantiation of client components.
Live bindings, enabling changes in the UI and in the data to be automatically propagated.
Command bubbling for codeless wiring of events in template-driven controls.
Data identity and association management for efficient and consistent client-server data exchanges.
Sending changes back to ADO.NET and REST data services.

In a nutshell, it is probably the easiest way to build a data-driven client application. Check this out:

<div id="peopleView" sys:attach="dataview" class="sys-template"
     dataview:dataprovider="{{ $create(Sys.Data.AdoNetDataContext,
                               {serviceUri: 'PeopleIKnow.svc'})}}"
     dataview:fetchoperation="PeopleIKnow"
     dataview:autofetch="true">
    <fieldset>
        <legend>
            <span>{binding FirstName}</span>
            <span>{binding LastName}</span>
        </legend>
        <img code:if="{{ Photo }}"
             sys:src="{{ 'Images/' + Photo }}"
             alt="{{ FirstName + ' ' + LastName }}" />
        <br />
        <input type="text" id="{{ $id('firstName') }}"
               class="editInPlace name"
               value="{binding FirstName}"
               sys:attach="inplace" inplace:cssclass="editing"/>
        <input type="text" id="{{ $id('lastName') }}"
               class="editInPlace name"
               value="{binding LastName}"
               sys:attach="inplace" inplace:cssclass="editing"/>
    </fieldset>
</div>
<br />
<input id="saveButton" type="button" value="Save" />

This creates a DataView that queries the PeopleIKnow.svc ADO.NET data service, and repeats the markup in the div over the data. The legend contains two spans that will respond to live changes to the data ({binding FirstName and {binding LastName}). The image will only be rendered if there is a photo to show (code:if). It builds the image path using a simple JavaScript expression: {{ ‘Images/’ + Photo }}.

The two input tags are augmented by a custom edit in place behavior (sys:attach=”inplace”) and are bound to the FirstName and LastName data columns so that any change to the value of the field will be propagated to everything that depends on the same data: the data itself of course but also the legend of the fieldset (see video below).

The save button is hooked to the following handler:

$addHandler($get("saveButton"), "click", function() {
    $find("peopleView").get_dataProvider().saveChanges();
}, true);

This handler is super-simple as it only has to call saveChanges on the data provider of the DataView. This is enough because any changes made in the input fields have been propagated to the client data model already, which tracked all changes and can build a simple JSON object to send back to the data service. Here is an example of the kind of JSON object that travels back to the server after I’ve changed Simon’s name through the UI:

{
  "__metadata":{
    "uri":"http://127.0.0.1:26402/Asp.Net_Ajax_Preview_4/
           PeopleIKnow.svc/PeopleIKnow(1)",
    "type":"PeopleIKnowModel.PeopleIKnow"
  },
  "ID":1,
  "FirstName":"Simon",
  "LastName":"Calvert",
  "Photo":"simoncal.jpg"
}

Here’s the application running:

<br/><a href="http://video.msn.com/video.aspx?vid=140e8d2a-8d01-49ab-b96f-77e219f93e40" target="_new" title="ASP.NET Ajax 4.0 Preview 4 ">Video: ASP.NET Ajax 4.0 Preview 4 </a>

The source code can be downloaded from here (contains code licensed under MS-PL):
http://weblogs.asp.net/blogs/bleroy/Samples/Asp.Net_Ajax_Preview_4.zip

But wait, there’s more in stock for the next preview… I’ll post more details about some of those features in the following weeks… Stay tuned…

http://aspnet.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=24645

JavaScript Behavior Sheets: an experiment

Bertrand Le Roy — Sun, 18 Jan 2009 08:53:00 GMT

Here’s a little experiment. I’m really after feedback on this one as I’m trying to decide whether this is a good idea. It’s also entirely possible somebody else did this before. That would be good feedback too. Anyway, here it is.

Despite its shortcomings, CSS has a number of features that make it very compelling. First, it decouples styling from markup. Second, its selector syntax is simple, yet reasonably powerful.

So we have semantic markup on the one hand, and styles on the other hand, and the only coupling between the two is the selectors in the stylesheet.

In Ajax applications, there is a third kind of entity in the mix, JavaScript behavior. There are of course ways to decouple the script behavior from the markup, which are usually referred to as unobtrusive JavaScript. jQuery also introduced back in 2005 a way to associate script behavior with the DOM using the same selector syntax that CSS uses.

But way before jQuery, Internet Explorer 5 enabled developers to specify behavior in stylesheets. You could do this for example:

.hilite { behavior:url(hilite.htc) }

This was a pretty neat idea at the time despite the challenges that came with it (in terms of performance for example), but it was never adopted by the other browsers despite having been submitted to W3C. The feature was never very widely used. There *is* a Firefox implementation of HTC behaviors but it doesn’t seem to have helped much in terms of adoption of this feature.

And one thing that bothers me with that idea is that while it does decouple behavior from markup, it also couples behavior to styling at least in location. Putting styles and behavior in the same file, in retrospect, looks like one step forward, and one step backwards. Sure, you could always use two separate files but the system did nothing to encourage you to do that.

What I’m trying to do here is separate styling, markup *and* behavior, leverage selectors in a nice, declarative way à la CSS and still work on all modern browsers. The coupling mechanism between style and markup (CSS selectors), is reasonably good and is already known by all Web developers so it remains a natural choice as jQuery showed clearly. Let’s see if one can express behavior in a way that is close to the already well-known CSS pattern.

In this implementation, it is possible to add event handlers and Microsoft Ajax behaviors from a “JavaScript Behavior Sheet” which can be embedded in the page or be loaded from a separate file using a modified script tag:

<script type="text/behavior" src="BehaviorSheet.jss"></script>

The contents of the tag or file is in the simple JSON notation for an object, omitting the curly braces:

"input[type=text].nomorethanfive": {
    click: function(e) {
        alert("You clicked input #" + e.target.id);
    },
    "Bleroy.Sample.CharCount": {
        maxLength: 5,
        overflow: function(source, args) {
            $(source.get_element()).jFade({
                 property: 'background',
                 start: 'FFFF00',
                 end: 'FFFFFF',
                 steps: 25,
                 duration: 30
            });
        }
    }
}

The top-level entities that can be found in there are the CSS selectors (note the quotes that are a departure from CSS notation but made the prototyping so much simpler). Each selector is associated with an object that contains event and object definitions.

The event definitions consist in the event name and the handler to associate with it:

click: function(e) {
    alert("You clicked input #" + e.target.id);
},

The implementation of this feature uses the new live events from jQuery. The result of that definition is that clicking on any input of type text with the class “nomorethanfive” will display an alert giving the id of the input that was clicked. That is a pretty efficient way to hook up events to multiple elements…

The behavior instantiation specifies the class to instantiate, “Bleroy.Sample.CharCount” and lists the properties, fields and events to set (here, the maxLength property is set to 5 and the overflow event is hooked to a function that flashes the element’s background yellow):

"Bleroy.Sample.CharCount": {
    maxLength: 5,
    overflow: function(source, args) {
        $(source.get_element()).jFade({
             property: 'background',
             start: 'FFFF00',
             end: 'FFFFFF',
             steps: 25,
             duration: 30
        });
    }
}

Here’s what the page looks like:

<\/embed><\/div>";" galleryimg="no">

I should point out that while the events will handle DOM mutations (such as adding new elements that match the selector) just fine, component instantiation won’t in this implementation, which is a limitation that is quite hard to work around in current browsers.

So here it is. All the code for this is available from the link below (contains jFade, code licensed under MIT: jQuery, and code under MS-PL: Microsoft Ajax and my own code), with some tests written with RST.js. So what do you think?

http://weblogs.asp.net/blogs/bleroy/Samples/BehaviorSheet.zip

UPDATE: interestingly, the intent behind Reglib by Greg Reimer is pretty close to this and he even uses the words “Behavior Sheets”.

UPDATE: Stuart Langridge did something very close to declarative events as they are done here back in November 2003. Thanks to Clayton for pointing that out. One should note that the limitations of the time are no longer preventing the technique from reaching its full potential.

PDC 2008 ASP.NET AJAX Futures talk available online

Bertrand Le Roy — Mon, 03 Nov 2008 18:10:00 GMT

The full 83 minutes of my PDC talk are available on the Channel 9 web site. You can watch the session online (using Silverlight) or download the video in a number of formats. Slides and source code for the demo are also available.

Watch online:
http://channel9.msdn.com/pdc2008/PC32/

Download MP4 / iPod, Zune, WMV or WMV-HQ.

Slides:
http://mschnlnine.vo.llnwd.net/d1/pdc08/PPTX/PC32.pptx

Demo code (contains MIT and MS-PL licensed JavaScript files):
http://mschnlnine.vo.llnwd.net/d1/pdc08/ZIP/PC32.ZIP

Another talk you may be interested in is Stephen Walther's. Stephen did an amazing job explaining how jQuery and ASP.NET Ajax work and fit together. The talk is very accessible even if you have no prior knowledge of jQuery: http://channel9.msdn.com/pdc2008/PC31/.

Client templates in MSDN Magazine

Bertrand Le Roy — Mon, 06 Oct 2008 04:48:16 GMT

My first full-length article in MSDN Magazine is out with the October issue and it’s about Microsoft AJAX client templates. Check it out…

http://msdn.microsoft.com/en-us/magazine/cc972638.aspx

Using client templates, part 1

Bertrand Le Roy — Thu, 31 Jul 2008 06:39:00 GMT

Last week, we shipped the first preview for the Ajax work we're doing in ASP.NET 4.0 under the simple form of a simple script file (release and debug versions). This should show how much emphasis this release puts on the client-side. As a matter of facts, I'll use a plain HTML file here instead of an ASPX file to make it perfectly clear that everything here (except for the web service that provides the data) runs on the client.

One of the scenarios we're trying to improve is updating parts of the page with new data without a postback (in other words, Ajax). That's a scenario you could implement in a number of manners. First, you could put an UpdatePanel control around your server rendering of the data and be done with it. That works, but is a little more chatty than you'd wish. It's all the chattier if you're dealing with a relatively large page with a lot of ViewState (in which case your small partial update pays a tax for the rest of the page that won't get updated). Even without ViewState, HTML isn't necessarily the most compact representation of your data.

It's usually more efficient, at least in terms of network traffic, to transmit pure JSON data and do the rendering on the client. Now doing that without a proper template engine is clearly not the best use of your time. You could do it either with DOM APIs and a hard to maintain mess of document.createElement and appendChild calls, or you could concatenate strings. The latter solution isn't much easier to maintain (how do you explain your HTML designer how to modify that stuff?) and is quite dangerous, in the same way that building a SQL query using string concatenation is. Nobody in their right mind does that anymore, right? I mean, ...right?

A good template engine should be highly readable, as close to designable HTML as possible (bonus points if it *is* a version of HTML that can be validated), it must have a good expression language and should perform well.

Our engine is using markup that can be (but doesn't *have* to be) valid XHTML, it uses JavaScript as the expression language (which means you don't have to learn yet another language), and we're quite pleased with how it performs. Let's have a look.

The page we're building here is a very simple example of getting some data from a web service (names and photos) and rendering that on the client. Here's what the template looks like:

<div id="peopleIKnow" class="sys-template">
    <fieldset>
        <legend>{{ FirstName }} {{ LastName }}</legend>
        <!--* if (Photo) { *-->
        <img sys:src="{{ 'Images/' + Photo }}"
             alt="{{ FirstName + ' ' + LastName }}" />
        <!--* } *-->
    </fieldset>
</div>

And here's what it renders like:

A few things to notice here.

The syntax to insert data, be it into text nodes or attribute values, is {{ expression }}, where expression is any valid JavaScript expression. The most common case will be that you want to inject a column of the current data item. For that case, you won't have to use anything like the server-side's Eval("columnName") or anything like Container.DataItem.columnName or $T.row.columnName. You just use the column name and that's it: {{ FirstName }}. This will just inject the value of the FirstName column into the markup. And of course when you need something more complex, like a combination of columns or some additional formatting, you have all the power of JavaScript and the Ajax library at your disposal.

The alt attribute of the image is one example of that: here, we're combining the first and last name. You could be tempted here to do something like alt="{{ FirstName }} {{ LastName }}". This won't work as the engine only accepts expressions for the full attribute and within text nodes (in the same way that on the server-side, <%# %> blocks have to be the full attribute).

The src attribute is one of those attributes that you have to prefix with the system namespace sys if you want to bind them (together with id, checked and a few others). There are several reasons for that. First, the template is part of the initial markup of the page. If you used src as usual here, the browser would try to download an image named "{{ 'Images/' + Photo }}", which will of course result in an unnecessary network request, a 404 and some junk in your server logs. Second, an Internet Explorer bug prevents us from reading the binding expression from the DOM (the browser won't give you the actual value that you set, but will always give you what it thinks is the right value, in this case the server-resolved url, which is useless to us). Finally, it can help achieving XHTML compliance (for which, I know, there is a small violation in this template in that the imag tag is missing an actual src attribute, which you can fix -if you have to- by including a src="about:blank" attribute). Some attributes don't accept just any value in XHTML. To summarize, if binding an attribute directly doesn't work, chances are prefixing it with "sys:" just will.

Last thing is this weird comment block, which we're using to mix JavaScript code into the markup. More precisely, we're doing conditional rendering of the image depending on the existence of the Photo field. There has been much debate over the use of a special comment-based syntax for this. We don't especially like using comments here as we agree that you should ideally be able to remove comments from markup without any impact. We decided to use them anyway because there wasn't a better solution that was still compatible with our XHTML constraint. We did consider admitting both  blocks and something like <script type="application/ms-template"> but that just seemed too verbose and it's rarely a good thing to have two solutions for exactly the same problem. It would be interesting to hear your comments on that, by the way...

So how do you transform this template into markup? Depends whether you're a component developer, in which case you're going to use the template APIs directly from your component, or if you're an application developer. I'll focus on the latter in this post.

To render the template with data, you simply instantiate a DataView control and feed it data:

var peopleIKnowView = $create(Sys.Preview.UI.DataView, {}, {}, {},
                              $get("peopleIKnow"));
PeopleIKnow.GetPeople(function(results) {
    peopleIKnowView.set_data(results);
});

In this code, we're using $create to create and initialize an instance of DataView on the "peopleIKnow" div tag, then we invoke the web service, and set the data property on the Dataview control from the callback function of the web service call. That's it.

Next time, I'll get into more details into what really happens when you instantiate a template, and I'll give some debugging tips.

The full code for this can be downloaded here: http://weblogs.asp.net/bleroy/attachment/6461783.ashx. Included script files are subject to the licenses for the Ajax Library and ASP.NET Ajax 4.0 CodePlex Preview 1.

Part 2 can be found here: http://weblogs.asp.net/bleroy/archive/2008/09/02/using-client-templates-part-2-live-bindings.aspx.

Dates and JSON

Bertrand Le Roy — Fri, 18 Jan 2008 20:47:00 GMT

JSON is a great data format and it's taken the Internet by storm for a number of good reasons. But because of a strange oversight in the EcmaScript specs, there is no standard way of describing dates in JSON. There's been a lot of discussion on this topic and it still remains a problem today.

In the Microsoft Ajax Library, we've tried a number of different approaches before we converged to the solution we're using today.

The first thing we tried was to inject Date constructors in the JSON string. This is a (very) bad idea for a number of reasons. First, it simply does not conform to the JSON specs. Second, any JSON parser that validates its input before parsing it will cough on such a thing. Finally, it establishes a precedent: why would it be allowed for dates and not for arbitrary types? This would just defeat the purpose of JSON.

The second approach, which is what most people settled on, is to adopt some string form of date and just interpret any string that conforms to this format as a date. We've been using "@1163531522089@" where the number is the number of milliseconds since January 1st 1970 UTC for a while (not super-readable) and some people are just using the ISO 8601 format. These are both almost acceptable compromises (if everyone agrees on them) but have a fundamental flaw which is that there can be false positives: what if you want to serialize the "1997-07-16T19:20:30.45+01:00" string, as a real string, not as a date?

To be perfectly honest, JSON Schema does solve the problem by making it possible to "subtype" a string as a date literal, but this is still work in progress and it will take time before any significant adoption is reached.

Our current approach is using a small loophole in the JSON specs. In a JSON string literal, you may (or may not) escape some characters. Among those characters, weirdly enough, there is the slash character ('/'). This is weird because there actually is no reason that I can think of why you'd want to do that. We've used it to our benefit to disambiguate a string from a date literal.

The new format is "\/Date(1198908717056)\/" where the number is again the number of milliseconds since January 1st 1970 UTC. I would gladly agree that this is still not super readable, which could be solved by using ISO 8601 instead.

The point is that this disambiguates a date literal from a string that looks like the same date literal, while remaining pure JSON that will be parsed by any standard JSON parser. Of course, a parser that doesn't know about this convention will just see a string, but parsers that do will be able to parse those as dates without a risk for false positives (except if the originating serializer escaped slashes, but I don't know of one that does).

Here's how you'd encode a string that contains the same literal I described above: "/Date(1198908717056)/".

Notice how a simple eval-based parser will just return the exact same string in both cases, but if you run a simple search for "\\/Date$(\d+)$\\/" and replace with "new Date($1)" before the eval (but after validation), you'll get the dates right in the final object graph. The string I described above will just go through that filter undetected and will remain a string.

We're pretty much satisfied with this solution to the date problem, but of course for the moment very few serializers and parsers support that convention. It would be great if this could become the consensus across the industry.

I'd love to read any comments you may have on that subject.