Tales from the Evil Empire

Bertrand Le Roy's blog

News


Bertrand Le Roy

BoudinFatal's Gamercard

Tales from the Evil Empire - Blogged

Blogs I read

My other stuff

Archives

How to choose a client template engine

Water lenses, (c) 2007 Bertrand Le Roy Disclaimer: I worked on the Microsoft Ajax 4.0 template engine, so my criteria are of course heavily influenced by our own design.

Templates are a data rendering method that server-side developers have enjoyed since the old days of classic ASP and PHP. The idea was quite simple (add code blocks and dynamic expressions directly into HTML markup) but it revolutionized web development, which before that relied on the opposite method (spitting HTML from CGI code).

On the client-side, the browser provides two ways to generate HTML: innerHTML and the DOM API. Template rendering is of course possible, but only using a JavaScript library. To be honest, one should mention XSLT here, which is standard and widely supported but whose somewhat unusual syntax has had limited success convincing web developers.

There are literally dozens of template engines available today, using many different markup conventions and algorithms. Template engines are not equal, they have different levels of complexity, different feature sets, different security models. It can be quite hard to pick one, and it really depends on the requirements of your application and also on taste.

I’ll try to go over a few things that you might want to check on a template engine. You will want to cherry-pick what’s important to you and what’s not so that you can pick the tool that’s best adapted to your problem.

1. Expression delimiter

A template system is adding new semantics to an existing language. This is not completely harmless as you have to give a new meaning to sequences of characters that were understood before as plain literals. That means that an escaping mechanism must exist to express the literal contents that were given a new meaning. For example, if the expression delimiters are <%= and %>, how do you express the “<%=” and “%>” sequences of characters as literals?

<%= and %> are by far the most common and familiar delimiters (they have been used by ASP, PHP, JSP, ASP.NET and many others), but they have one flaw that couldn’t be seen back in 1996: XHTML compliance. Granted, you may not care at all about XHTML. If you don’t, just skip this. But if you do, any template engine that uses <%= %> simply can’t have template code that is XHTML-compliant. Now this may still not be a big deal if you don’t care that the template code is compliant, but only care about the generated markup.

But there is another reason why <%= %> is a bad choice: it’s conflicting with the server-side. A client-side engine must be able to coexist peacefully with whatever server-side technology you’re using. If your server technology of choice is using <%= %> like it very likely does, it will be conflicting with your client templates. The server-side engine will kick in when it sees these delimiters and will fail one way or another.

We chose {{ }} as the expression delimiter (and also {expr } for extensible markup expressions such as {binding }), which happens to be the same choice that Django made (and Dojo, as they implement Django on  the client-side, and by the way I don’t know how they manage conflict with server-side Django; if you know, drop me a comment). It works fine with XHTML and it doesn’t conflict with ASP or PHP. If you want to express the “{{“ literal, we don’t provide an escape sequence per se, but you can still do it by writing {{ “{{“ }}.

On a final note, there are a few template engines that do not have expression delimiters but instead rely on microformats. While this absolutely preserves the HTML semantics, it’s often clumsier to use and quite limiting. It’s a matter of taste, but microformats also can look like markup within markup.

2. Expression language

In the simplest cases, you’ll want to inject a simple data field into the template, like {{ Price }}. But many times, you’ll want to go beyond those simple cases and embed a more complex expression into the markup. For example, you might want to display that price with two decimal places, like $42.00. In order to handle that case and more complex ones, it is necessary to have a full expression language.

In the same way, if the template engine allows for code blocks to introduce control structures such as conditional execution or loops, it needs a full programming language.

Some template engines, such as Django, introduce their own new language. Others, such as ours, just use what’s already available and familiar: JavaScript. In addition to being familiar, it eliminates the need to write a parser and interpreter, which reduces library code size and gives better performance.

3. Protection against injection attacks

If you don’t know what an injection attack is… wow. Just wow. Go read this, now. Seriously.
http://weblogs.asp.net/bleroy/archive/2004/08/18/please-please-please-learn-about-injection-attacks.aspx

The frightening thing about almost any single one of the client template engines out there is that they have no mechanism in place to protect against injection attacks, which puts the responsibility to encode and check contents in the hands of the application developer. That’s you.

Most of the engines out there are using a very simple algorithm to build HTML: array joining, and then injecting the result using innerHTML. This is very similar to building SQL by concatenating strings: in a nutshell, you shouldn’t do that. In the same way that SQL is better built using parameterized queries, HTML is more securely built using the DOM API which just eliminates the need for encoding.

Even when using encoding or the DOM API, there is a number of attributes that still present a fair amount of danger if user data gets injected in them. For example, if you bind the href attribute of a link to a piece of user data, no amount of encoding will protect the application against a user injecting “javascript:doSomeEvil();”.

To prevent that kind of attack, we white-list the protocols in all known URL attributes to relative URLs, http and https. It is possible for the application to extend this white list as needed (available in Microsoft Ajax 4.0 Preview 4 and later).

One thing to understand is that the dangerous contents here is the data, not the template itself, which is trusted as application code is.

4. Template location

Depending on the library, the template can be embedded in comments, in script tags or just be part of the DOM. It can also be included from a separate file.

Putting the template in comments or script tags easily hides its markup from initial rendering, but it makes it harder to design with existing tools and puts it out of the reach of markup validation tools.

Including from a separate file adds one more request to the server, which might be a problem or not: it might also improve performance by allowing partial caching. The best is to have a choice here, so I’d reject an engine where external files is the only possibility, knowing that all other engines allow external files with minimal effort by feeding a simple XHR’s result into the system.

Instead of using script tags or comments, we chose to embed the template as real HTML contents anywhere in the document and just hide it from initial rendering using CSS. This makes the code designable using any existing tool, and enables us to use the browser’s native HTML parser to understand the structure of the markup.

5. Hooking up events and instantiating components

In a modern Web application, you won’t just create plain HTML, you’ll want to hook up events and instantiate components. The template engine might let you do this inline, which is obtrusive but convenient, or it might require you to enrich the DOM after template instantiation through code, or it might give you a choice between the two.

One thing to look for is how much knowledge of the template markup’s shape must be injected into the code in order to enrich the DOM.

For a picture of the different possibilities using our engine, read this post:
http://weblogs.asp.net/bleroy/archive/2008/11/28/instantiating-components-on-template-markup.aspx

6. Live bindings

One big advantage of client rendering is that the data remains under raw data form until it reaches the template engine. That means that you have a representation of the data on the client. This in turn means there is an opportunity to monitor changes to the data and to bind UI directly to it, eliminating round-trips to the server along the way.

We call this ability live bindings and it makes for very responsive applications that are still easy to build.

7. Constraints on markup shape

One thing you should try with any template engine is to repeat a row in a table, using IE. This may sound trivial, but innerHTML doesn’t work on IE in a number of places, which puts a constraint on the markup that you need to write to render certain shapes of HTML. There is almost always a workaround but it can be more or less painful to implement. Try and compare, see what you find reasonable.

8. Ease of data field access

This one is quite simple. Some template engines force you to access the current data item’s field through some special syntax such as “$T.price”. In our engine and several others, you just write “price”.

9. Readability

This one is quite subjective, but when choosing a template engine, just look at a few samples. Does the code hurt your eyes? Can you understand it right away?

10. Performance

A template engine must be able to render a reasonable number of data items with a reasonably complex template under the perception threshold. What’s “reasonable” pretty much depends on your application design, but I’d say a hundred items in a typical grid layout should be hardly noticeable. So before you choose, do a mock-up of your design and data and try it. Increase the number of items and see where rendering starts to get noticeable. Usually rendering performance is not linear so this should give you a good idea of how far you can go with each engine. Better to determine that before you invest heavily in one engine.

11. XHTML compliance

As I said above, this is one you might not care about. Don’t leave yet, it actually goes both ways, as an engine, by enforcing XHTML compliance, might potentially get in the way of writing HTML the way you want.

The lowest level of compliance is that the rendered output is compliant. Being template engines, most if not all engines enable you to create compliant or horribly broken markup, depending on your own style. What it must absolutely not do is prevent you from generating compliant markup.

The second level of compliance is that the template code itself, before data is injected, is compliant. This is much less common. Even less common is that the compliance requirement does not impose too strong a constraint on the markup you have to write.

In other words, a good template engine lets you write XHTML but doesn’t force you to.

12. Nested templates

Rendering a template within a template might look like a whacky scenario at first if the engine understands control structures such as loops. In reality, if you include something like live bindings into the mix, you will want for example an inner template to be re-drawn when the outer data item changes. This will be much easier to achieve with nested templates.

So if this is a scenario that you care about, make sure that your engine handles that case gracefully.

In conclusion

There are many engines to choose from, and many criteria to check. I’m quite convinced that our own offering is pretty solid for all these criteria and offers a good compromise of features against simplicity and performance.

There is another engine that I quite like because it’s an impressively small bit of code that remains useful, and that is John Resig’s micro-template engine, which is only 20 lines of code. Like our own engine, it “compiles” the template into a reusable function that takes the data as a parameter. But unlike ours, it has no security mechanism and has issues with some markup, which may or may not be an issue for you. It’s a different kind of compromise that puts code size and simplicity before any other consideration.

I hope this helps in making a choice.

A good resource on client templates:
http://ajaxpatterns.org/Browser-Side_Templating

John Resig’s micro-templates:
http://ejohn.org/blog/javascript-micro-templating/

ASP.NET 4.0 Ajax Preview 3:
http://www.codeplex.com/aspnet/Wiki/View.aspx?title=AJAX

UPDATE: added performance and XHTML compliance to the list from Dave Reed’s feedback, and nested templates from Keith’s.

Comments

John Pennington said:

Bertrand, I couldn't help but notice a reference in this post to Ajax 4.0 Preview 4 in the "Injection Attacks" section. I understood that P4 was originally targeted for release in December 2008. Do you have an update for when P4 will become available on CodePlex? Thanks and keep the posts coming.

# February 5, 2009 8:32 AM

Chris said:

Of all the template languages I've used in Rails, CakePHP, Catalyst, Merb, Django, etc, Django is the only one I've encountered where the variables you put in the templates are all automatically encoded/escaped by default. You could tell it to NOT automatically encode values on a variable by variable basisc if you needed to.

It would seem that the escape-by-default should be the default behavior on a lot more templating engines.

# February 5, 2009 9:48 AM

Bertrand Le Roy said:

@John: soon. Almost done. :)

@Chris: it's great to encode by default, and that's what we're doing in our client templates, but it's important to keep in mind that it's not enough. For example, it doesn't protect against the javascript protocol being injected into a url attribute.

# February 5, 2009 1:04 PM

InfinitiesLoop said:

Actually, there's an important difference between "enocding automatically" and not needing to encode in the first place. Because our solution uses DOM apis to build the instance nodes, it isnt actually enocoding anything, because it doesn't need to. An engine that needs to encode, even though it does it automatically, will suffer from that performance/memory usage hit as well as be vulnerable to attacks on the encoding/decoding mechanism itself (if any).

# February 5, 2009 4:00 PM

Bertrand Le Roy said:

@Keith: yes, nested templates are an important feature, which we're handling fine.

# February 5, 2009 6:44 PM

Mike said:

Great post, it helps me evaluate template engines before I even knew what to look for. I did know that I wanted pure javascript. Looks like the ASP.NET Ajax templating solution is a good one. How large is it to include on a page?

# February 9, 2009 6:51 AM

mckamey said:

First of all, thanks Bertrand for the good thought that went into this post.

Disclaimer: I built the JBST client templating in JsonFx.NET, so my responses are equally biased.

1. Expression delimiter:

I find that the familiarity of "<%=" and "%>" makes it the most natural to work within.  It can be escaped the same way they have since SGML, with character entities: "&lt;%=" and "%&gt;". All the JsonFx documentation has embeded examples writing in the templates.

XHTML compliance? This assumes that your templates are stored directly in the markup. JsonFx precompiles JBST controls to pure JavaScript. This means that the resulting templates themselves are built from XHTML but may be compacted, compressed and cached just like any other JavaScript file.

Server-side code: This hasn't been an issue in JsonFx either because JBST controls are built just like ASP.NET UserControls which are then attached to a page rather than residing within them.  Again caching was the driver on this one.

2. Expression language:

I wholeheartedly agree. Adding another language to the mix is unnecessary. JavaScript has its own beauty and fits well within a template solution. Again familiarity is a design choice.

3. Script injection protection:

The only way this is an issue is if user data is being treated as markup. JsonFx doesn't combine data as markup but as encoded text by default. There exists a mechanism to treat user content as markup, but the template compiler also doubles as a highly configurable HTML parser which can scrub user data first.

4. Template location

JsonFx uses built-in Visual Studio tools to precompile and deploy templates as cachable attached scripts.  These can be combined (at build-time) with the scripts that already must be deployed, so the result is actually *fewer* requests to the server.  I have yet to find a real world situation where this is worse than embeded.

5. Events

I agree with the choice on this one. Sometimes it can actually improve the readability to attach events directly. In all reality, unobtrusive JavaScript loses its utility when the DOM elements aren't even available without JavaScript. At that point it becomes pure academia. JsonFx provides mechanisms for both.

6. Live bindings

Amen! This is the whole point of client-side templating. The other choice is what's called HTML message pattern which is easy to build but far, far less efficient.

7. Constraints on markup shape

Tables in IE aren't the only oddity of IE DOM, but they certainly are one of the worst (stylesheets are up there as well). JsonFx uses full DOM methods to create the data-bound templates which avoids the pitfalls of innerHTML.

8. Ease of data field access

Access to data is something which I toiled over when creating JsonFx. My design choice was to leverage the "this" keyword since in global usage it is of little use being bound to the global "window" object.  JsonFx allows access to template specific functions, data and meta-data via "this.".  Those extra characters allow references to data like "price" to not collide with otherwise globally accessible objects and data. Pure ease-of-use is a double-edged sword that can have painful side effects if not careful.

9. Readability

Amen, amen, amen (subjectively, of course).  JsonFx looks/feels just like ASP/JSP/ASP.NET UserControls.

10. Performance

This is extremely important. This is the thing that drives someone to choose client-side templating, so it shouldn't be the part that breaks down first.

11. XHTML Compliance

I agree that XHMTL compliance should not be prevented, nor forced. Again, though, this is very ASP.NET AJAX design-centric. If your templates never sit in markup and they are rendered with JavaScript/DOM then this is moot.

12. Nested Templates

Agreed. No real world scenario would be maintainable without nested templates. The reuse of sub-templates makes life a lot simpler.

In conclusion...

It is interesting that virtually all of these factors are very ASP.NET 4.0 AJAX centric. For other templating systems such as JsonFx.NET, many don't even apply. I encourage those looking at ASP.NET AJAX to take a look at JBST development within JsonFx: starterkit.jsonfx.net

# March 8, 2009 4:21 PM

Mic Cvilic said:

@Keith:

If you have time, could you just drop a post to our discussion group about the recursion problem you faced. I thought we had it working well.

The address is:

groups.google.com/group/Pure-Unobtrusive-Rendering-Engine

Thanks for your reference to PURE.

# March 30, 2009 3:35 AM

Chriss said:

How do you whitelist, I'm trying to link into spotify but with no luck at the moment.

# November 5, 2009 3:07 PM

Bertrand Le Roy said:

@Chriss: the two protocols that are whitelisted by default are http and https, but you can push a new one into Sys.UI.Template.allowedProtocols.

# November 5, 2009 3:37 PM

Mic said:

@aefxx Could you describe what makes jQote2 way more powerful than PURE?

Thanks,

# July 11, 2010 9:36 AM