High-performance XML (III): returning well-formed XML from WebServices without XmlDocument

Note: this entry has moved.

Recently, Matt Powell wrote about returning XML from webservices, and I certainly agree with him that returning it as an opaque string is really bad. Later on, Matevz Gacnik suggested a couple points to consider when to choose one or the other. Matt continued his rant  this time tackling at the heart of the problem: why do you need to load a full-blown DOM just to get the nice XML returned from the webservice? At this point, I felt I should kick in.

You see, one of the greatest things about having a fully OO platform is that you can easily complement what's missing by simply inheriting a couple classes, plugging your stuff in the infrastructure. So, let me state it clear and loud: you don't have to load an XmlDocument to return well-formed XML from webservices. Let's see how this is accomplished.

As you may already know, the ASP.NET WebMethod framework uses the XmlSerializer to convert arguments and return values to their XML representations. When your webmethod returns an XmlDocument, it will serialize the XmlDocument.DocumentElement, effectively the root element. If you return an XmlNode, it will simply serialize it. So, you could either return an XmlDocument *if you already have it at hand*, or an XmlNode (this could even be the DocumentElement of the previous one). The client will see the same WSDL and the Visual Studio Add Web Reference feature will generate a proxy returning an XmlNode in either case. So, *both* methods below result in the same proxy class:

[WebMethod] 
public XmlDocument MyDocumentMethod()
{
//...
}



[WebMethod] public XmlNode MyNodeMethod()
{
//...
}


// Proxy code
MyService proxy = new MyService();
XmlNode xml = proxy.MyDocumentMethod();
// or xml = proxy.MyNodeMethod();

When the XmlSerializer used to build the SOAP Body for our return value realizes it's an XmlDocument/XmlNode, it will simply call its WriteTo(XmlWriter w) method. So, what can we do with all this knowledge? Well, we can leverage it to avoid loading full DOMs when we have other (better IMO) representations such as an XmlReader or an XPathNavigator. The process is simply to create a special-purpose XmlNode-derived class that will serve as our trojan horse (very appropriate now that Troy is in vogue ;)) into the webservice serialization infrastructure.

Implementation

Implementing the XmlNode.WriteTo() method with an XmlReader is all too easy:

private class XmlReaderNode : SerializableNode 
{
private XmlReader _reader;
private bool _default;

public XmlReaderNode() {}
public XmlReaderNode(XmlReader reader, bool defaultAttrs)
{
_reader = reader;
_reader.MoveToContent();
_default = defaultAttrs;
}

public override void WriteTo(XmlWriter w)
{
w.WriteNode(_reader, _default);
_reader.Close();
}
}

Note that we need to move the reader to the actual content, because we don't have to serialize the document declaration again, because the result is placed inside the SOAP body. We'll get to the base SerializableNode class in a minute. Basically it overrides everything and throws NotSupportedExceptions.

Implementing the XmlNode.WriteTo() method with an XPathNavigator is also easy, as we can take advantage of the Mvp.Xml project XPathNavigatorReader:

private class XPathNavigatorNode : SerializableNode 
{
private XPathNavigator _navigator;

public XPathNavigatorNode() {}
public XPathNavigatorNode(XPathNavigator navigator)
{
_navigator = navigator;
}

public override void WriteTo(XmlWriter w)
{
w.WriteNode(new XPathNavigatorReader(_navigator), false);
}
}

Note that in both cases, we use the built-in infrastructure by calling the XmlWriter.WriteNode() method. Specially in the case of the XmlReaderNode, that means we're never building an in-memory DOM. In the case of the XPathNavigatorNode it means that we're working with the existing in-memory infoset, and we're not loading another one for the serialization.

The only "trick" in the base SerializableNode is in its constructor, because the base XmlNode and XmlElement don't allow empty constructors. The workaround was to create an element with a dummy name and an empty owner document:

private abstract class SerializableNode : XmlElement 
{
public SerializableNode() : base("", "dummy", "", new XmlDocument()) {}

public override XmlNode AppendChild(XmlNode newChild)
{
throw new NotSupportedException(SR.XmlDocumentFactory_NotImplementedDOM);
}
//...all other members...
}

At this point you may be wondering why all these classes are private. Well, in order to isolate the user from the internal implementation details of these classes, I prefer to create a factory class that simply returns XmlNode values:

public class XmlNodeFactory 
{
private XmlNodeFactory() {}
 public static XmlNode Create(XPathNavigator navigator) 
{
return new XPathNavigatorNode(navigator);
}
public static XmlNode Create(XmlReader reader)
{
return Create(reader, false);
}
public static XmlNode Create(XmlReader reader, bool defaultAttrs)
{
return new XmlReaderNode(reader, defaultAttrs);
}
//...private inner classes follow...
}

Usage

Now, let's get back to your webservice code. Most probably, specially if you have been reading my weblog and many others that advice against the XmlDocument, you'll already have an XmlReader or XPathNavigator you got from your business classes or your data access layer. With that at hand, you can simply use the following code:

[WebMethod] 
public XmlNode GetFromNavigator()
{
XPathNavigator nav;
// Get your navigator...
return XmlNodeFactory.Create(nav);
}
[WebMethod] 
public XmlNode GetFromReader()
{
XmlReader reader;
// Get your reader, maybe even from SQL Server?...
return XmlNodeFactory.Create(reader);
}

Note that I mention getting the reader from SQL Server. If you look at the XmlReaderNode code shown in the previous section, you'll notice that once the WriteTo() serialization method is invoked, the reader is closed.

The interesting thing in the XPathNavigator case is that you can position the navigator on the node you want to return, and have only that "fragment" serialized. For example:

[WebMethod] 
public XmlNode GetFromNavigator()
{
string xml = "<customer><order>25</order><info>Daniel Cazzulino</info></customer>";
XPathDocument doc = new XPathDocument(new StringReader(xml));
XPathNavigator nav = doc.CreateNavigator();
//Move to the customer node
nav.MoveToFirstChild();
//Move to the order node
nav.MoveToFirstChild();
return XmlNodeFactory.Create(nav);
}

The returned data will be only the <order>25</order> element (and any children it may have). Cool, huh? That's XPathNavigatorReader courtesy ;)

The full project source code belongs to the Mvp.Xml project and can be downloaded from SourceForge. Enjoy!

Check out the Roadmap to high performance XML.

9 Comments

  • Great solution, you XML think-tank!



    The downside is that this one does not ship with the current framework.

  • whew - I need to get a blog :)



    ---------------



    A few quick notes - not that I'm defending the whole return a string as xml technique – but these are the remaining questions I have…



    -If you serialize a dataset (i.e. return type is DataSet), it simply writes out xml, and fat xml at that (not relevant to this post – see Matt’s rant)

    -If you return dataset xml (or the XmlReader) to the client, doesn’t the xml get processed twice (once by the SOAP pipeline, once when rebuilding the DataSet)? It seems to me passing one huge string as an element within the SOAP message would be far less resource intensive than parsing xml within the SOAP message.

    -I love the idea of returning the XmlReader – I can create a custom XmlReader on top of a DataReader so effectively the soap pipeline would be pulling directly from the database - NICE



    -That being said, I’ll probably test some of these scenarios out here in the next couple of weeks and report back – this issue is CRITICAL to most enterprise applications.



    ---------------------------



    As a little background on where I’ve been coming from -- the main problem I’ve been dealing with is getting data to a client application over web services. The main problem here is most client side controls in .net are best used by binding to datasets. So the pipeline in a client/server app would be:

    -Execute a query

    -Fill dataset with result

    -Bind to a control



    In webservice mode, I need to get the DataSet (or more commonly, several datasets) across the wire

    1) as quickly as possible

    2) with as little memory/cpu as possible, as long as it doesn’t kill 1



    The solution I’ve evolved to over time goes something like this:



    -(skipping the mechanics of calling to get data…)



    -Instantiate a datareader

    -Manually write xml schema to a StringBuilder (that matches the DataSet schema, so it can be loaded)

    -Manually write xml data (with columns as attributes instead of elements) to a StringBuilder (that matches the DataSet schema…)

    -Return the StringBuilder result



    -Return the string (probably with other objects) across the wire



    -Reconstitute the DataSet on the client - ds.ReadXml(new XmlTextReader(new StringReader((string)de.Value)), XmlReadMode.ReadSchema);





    This technique

    1) Is an order of magnitude faster than returning a DataSet directly from a web service call (depending on size, of course)

    2) Enables me to return a DataSet (well, the dataset data) as part of another return type)



  • OK Daniel,

    At least I took the time to rewrite my old article using an XmlTextWriter. I feel better now! Thanks.

  • Daniel,



    The last example contains some bad xml. You have and ending &lt;/root&gt; that should be a &lt;/customer&gt;.





  • You're right Paul! Thanks!

  • I think you're missing the point here. When you get a string from Oracle, I assume you get a string with the XML. In Yukon (next version of SQL Server), you will actually be able to get this as a more programmer friendly type, such as an XmlReader.

    Now, if you followed by post, what you're actually returning from this webservice will be an *escaped* version of the XML, if you return the string directly.

    Therefore, your xmlDoc.loadXML will NOT work. All you have in the client is an *escaped* version which is not a well formed XML at all.

    But you're right, if you already have the string, it may be easier to just return it. Like I said, however, this reduces the usability of your results as they have to be unescaped prior to use. Likewise, your server-side code will also have to escape the string before returning the SOAP response.

    So, it's true you avoid parsing. It isn't true that you avoid *all* processing.

    I prefer to stick to well-formed results every day.



  • But then if you return an XmlReader from Yukon you pay the cost of creating and maintaining an XmlReader in both sides rather than a simple string. Mind you, an XmlReader that serves no purpose at all in this case, it only needs to be passed along to the client.



    When using strings, of course the Web Service will escape the string implicitly for you. But you're forgetting that the consumber will unescape it right back, just as implicitly.



    Therefore, loadXML will work just fine.



    Escaping and unescaping of the string is done internally by the Web Service implementations in both sides, there's nothing you actively need to code yourself to get the original string back. When you retrieve the string (via the first argument sent to the callback function in the above case) the result is the exact original string, after unescape (any other implementation would be silly).



    And xml-escaping a string is much simpler than even a very light partial xml parser.



    Now, don't get me wrong, I fully understand where you come from and the &quot;purist&quot; argument of returning the &quot;correct&quot; datatype from a (Web)Method. But just like anything else in life, that attitude is context-sensitive and should be considered on a case-by-case basis.



    Thanks for the wonderful work you're doing with this series, it's been quite helpful.

  • &gt; But then if you return an XmlReader from Yukon you pay the cost of creating and maintaining an XmlReader in both sides rather than a simple string.



    Em... nope. Any webservice just returns a string, the SOAP response. It's up to the client toolkit you use how it's represented. If you return the XmlReader (actually using what I've shown in this article, as the reader is not serializable), a .NET client will receive an XmlNode, therefore, you don't even have to bother to loadXML ;)... For a javascript client, it's just a string like yours, but doesn't need to be escaped. So, what's the drawback? I don't know what you mean by &quot;maintaining an XmlReader&quot;...



    &gt;And xml-escaping a string is much simpler than even a very light partial xml parser.



    But then you do loadXML, so you need to parse it anyway in order to make anything useful. So, why to xml-unescape if you can avoid it altogether? You can't avoid parsing XML anyway...



    You see, it's not about purism. Suppose at some point you need to add integration with other systems. Suppose you need to add business events routing and notifications through external systems (does Biztalk ring a bell?). Suppose you need output validation, transformation, security, etc.

    If your output is just an opaque string, you're losing all that. Worst, even if you don't need it now, you're precluding future innovation and functionality. Just for the sake of avoiding returning the *correct* data type? I prefer to stay on the safe side.

    But that's just me ;)

  • BTW, I'm glad you find my series useful :D

Comments have been disabled for this content.