XLinq: is XML embedded in a host language a good idea or a terrible one?

Note: this entry has moved.

During the MVP Summit, and through various articles and blog posts, Microsoft has been trying to get community feedback on the proposed feature of integrating XML as a first-class concept into a core language (such as VB.NET). It seems so natural, right? After all, they are not the first ones to propose such a thing.
You're dealing with lots XML in your application, what else could you ask than to be able to do something like the following?:

Dim CountriesWithCapital As XElement = _ 
  <Countries> 
    <%= Select <Country Name=(Country.Name) 
                        Density=(Country.Population/Country.Area)> 
                 <Capital> 
                   <Name><%= City.Name %></Name> 
                   <Longitude><%= City.Longitude %></Longtitude> 
                   <Latitude><%= City.Latitude %></Latitude> 
                 </Capital> 
               </Country> _ 
        From Country In Countries, City In Capitals _ 
        Where Country.Name = City.Country %> 
  </Countries>

I see at least a couple drawbacks with this approach:

1. VS already has a full-featured, very responsive and cool XML editor that includes intellisense powered by schemas, allows XSLT transformation debugging, etc. We would need to have at least an equally capable inline XML editor in place in order to be really productive
2. Now the “template” for the “rendered” XML is compiled into the application. If I need to change anything about that XML layout, I’ll need to recompile the application.

Of course, this does not mean that we should stick to using XmlWriter and XmlDocument to create XML. That’s just insane and much harder to maintain. But there is another alternative, which I think is better.

Text templating is an old technique. The reason why the syntax above looks so familiar is that it’s mostly classic ASP, and people have been using a similar technique for generating not only XML or HTML, but all kinds of text assets (such as SQL statements). But so far, no general purpose text template rendering engine has been ever been available from Microsoft. Until now.

The DSL tools and GAT come with such an engine code-named T4 (Text Templates Transformation Toolkit, kind of a successor of a previous version code-named T3 which I developed quite some time ago for patterns & practices and bundled in previous versions of GAT). This engine allows you to author very similar XML building code but in a standalone file, that you can later “render” with a given context:

<#@ Template Language="C#" #>
<#@ Property Processor="PropertyProcessor" Name="Customer" #>
 
<?xml version="1.0" encoding="utf-8" ?>
<Customer Name="<# this.Customer.Name #>">
<# foreach (Order order in this.Customer.Orders) { #>
 <Order>
  <Id><#= order.Id #></Id>
  ... etc ...
 </Order>
<# } #>
</Customer>

You can of course choose the language you will use for the template so-called control code, and have properties automatically created by specifying that Property template directive. If you had chosen VB as the language, and were using VB9 runtime/compiler, I suppose nothing would stop you  from having almost *exactly* the same code shown at the beginning. Now, forget about T4, as it’s a tool part of an SDK, probably not well tested outside of the scope it was designed for (code generation at designtime), etc. Imagine if you had such an engine but as part of the .NET framework itself. Imagine you had deep support for intellisense in VS for the control code snippets. There are several benefits to such an approach:

1. One of the main benefits is that now the language used to author the control code is a “plug-in” to the template editor. This ensures that the template editing experience will be consistent independently of which .NET language you decide to code the control code snippets in. The opposite case happens if you embed the template code (XML in XLinq case) into a host language: all host language editors will need to provide this feature, and we have already started seeing that some .NET languages will support such a feature (probably VB9) whereas others will probably not (most likely C#). Even if they all agree to implement the editor and required language features, third-party languages will still be out of the game unless they spend substantial time adding the feature in a way that is consistent with everyone else’s. In a T4–like approach, pretty much all you would need is a CodeDomProvider.
2. Given 1., the designers of the template editor will need to design it with pluggability in mind. In fact, this editor will easily become the host for all of the other editors in VS, including the XML editor, SQL editor, C#/VB, ASP.NET, etc. Once this is well done for any one of them, integrating the others will be substantially easier than trying to extend each and every language to support “deep integration” with any of those “languages”.
3. This will also maximize the productivity of developers that are proficient with this template language. Now they will be able to comfortably generate any target source asset using the same tool.
4. Mixing the “rendering” code (XML in this case) with the model and controller code makes for poor maintainability. You should think of the XML being generated as a view over a model, resembling very much the MVC pattern. There’s a very interesting paper regarding that. It's already a bit frightening to have the full .NET power at hand inside the template control code... but having it fully intermixed with the M-C code will probably lead to spagethi-like code that we thought we finally got through with ASP.NET. In this regard, I see it as a door for a very likely huge step backwards. (do you really think people will NOT generate HTML or other rendering code with it?)
5. As the templates themselves can be made to inherit from a given base template class (via an Inherits="MyNamespace.MyBaseClass" attribute on the template directive), you can maximize reuse of control code and rendering helpers, as well as get intellisense driven by this custom base class too (together with whatever the so-called directive processors provide to the template class).
6. If you take this paradigm further enough, you could even end up with an editor that is much like the ASP.NET one, with custom rendering “controls” which can be templatized (just like in ASP.NET), use binding expression, etc..
7. Finally, but not less important, is that you can now evolve your “rendered” views without recompiling your entire application. If missed an attribute that is available on the model, but was not needed previously on the rendered/output XML, you can add it simply by changing the template.

I think this is a very interesting area, but I believe Microsoft should strive not to provide another technology that could potentially bring back the nightmares we lived with ASP in the past. If it can be missused, it *will* be missused.

If it were for me, I’d limit the features available on the control code snippets to just control and rendering code: if..else, for/while loops, value retrieval (<#= foo.ToString #>), and that’s it. No object creation, no method calls (as they could have side-effects).