DonXML Blog : XML

Announcing the NJ SQL Server User Group

DonXML — Mon, 21 Nov 2005 15:05:00 GMT

It has been a long time coming, but NJ will finally see the return of a SQL Server User Group. I’ve been trying to get this group going for about a year now, and the biggest hang-ups has been trying to get a location, and finding volunteers to help run the group. Well, we solved both issues with the help of the great folks at SetFocus (who run the Northern NJ .Net User Group N3UG). My wife, Melissa (aka SQL Diva), is going to run the user group, and SetFocus will let us hold the meetings in their facility in Parsippany, NJ (which is very close to where I live, and makes it possible for Melissa to run the group). The meeting will be held the 3rd Tues. of the month (a week after the .Net User Group), and will start on Jan. 17th. Informal networking begins at 6PM and the presentation starts at 6:30 (until about 8PM).

This SQL group will not be like most other SQL groups, as it is dedicated to the Data Tier Developer (sprocs, integration services (aka DTS), Analysis Services (aka OLAP), Reporting Service, etc.) , and will not contain presentations relavent to the Support DBA (back-up, recovery, etc.). The user group will join both PASS, and INETA, and we plan on taking advantage of the INETA Speaker’s Bureau. I’m going to be doing the first presentation, Intro SQL Server 2005. If you are interested in presenting, please contact Melissa, via the website www.njsql.org or her website, www.sqldiva.com. The NJ SQL site isn’t 100% complete. I’ve still got a lot of skinning of CommunityServer to do, and we have a lot more content to put up, but we wanted to let everyone know about the group, ASAP.

The preceding blog entry has been syndicated from the Don Demsak’s All Things Techie Blog. Please post all comments on the original post.

XML Tools I’d Like To See Built For Visual Studio.Net

DonXML — Fri, 17 Dec 2004 02:46:00 GMT

While at this year’s XMLDevCon I came up with 2 VS.Net add-ins that I’d wanted to write and release as open source for .Net. I was withholding the ideas figuring that I would find time to write them, but it has been a couple months and I haven’t even begun work on them. So, I’ll publish them here, and hopefully someone else will help take up the gauntlet.

In .Net 2.0, XSLT will compile down to IL (and you can debug it in VS.Net). The add-in I’d like to see is one that would let you compile your XSLT right into your .Net project and not need (a) separate XSLT file(s). Yes, I know that you may not want this in all circumstances, but there are a lot of times that my XSLT is pretty much static. I’ll pass in parameters, just like any XSLT, but isn’t a shame that I have to always treat XSLT like a resource file, and have to do all this compiling and caching on the fly. Wouldn’t it be cool to just compile it into your project when you wanted it? An extension of this idea would not to just include the IL in your project, but use something like Lutz Roeder’s Reflector to generate your favorite .Net language and make that part of your project (effectively making XSLT a code generation language within .Net). Some things are just easier to write in XSLT.

The other add-in is really just an extension of the XSLT idea, and that is to do the same thing for XML Schema. Why does the XML schemas always have to be a separate file, and dynamically “compiled” and used. Actually, compiling XML Schema down to IL, and including it in a project would probably be used more often then someone would use the XSLT add-in. And, if we use Reflector, you could even add some validation rules that couldn’t be done well in XML Schema (think Schematron). I guess that would turn XML Schema into a code generation tool, but I don’t think that is so bad.

The preceding blog entry has been syndicated from the DonXML Demsak’s All Things Techie Blog. Please post all comments on the original post.

Do We Love XSLT Because It Is Difficult To Learn?

DonXML — Mon, 13 Dec 2004 01:06:00 GMT

M. David (on his XSLTBlog) asks why some of us are drawn to XSLT and has a great quote (well sort of quote) from Tim Bray from a late night drinking session at XMLDevCon:

to understand XSLT you must first reach up with both hands and grab the two halves of your brain peeling each back until you have successfully turned your brain completely inside out

M. David thinks that it is just because XSLT is hard to learn, and we love to torture ourselves. Me, well I think that some of us realize (in the deep dark recesses of our brains) that procedural programming (aka what most people call programming) is preventing us from reaching out full potential and that some form of declarative programming is the future. XSLT reminds us that there is more then just procedural programming out there, and we really have to stop and open our minds to other forms of programming. I think that once we get beyond the coding everything procedurally mentality, we will finally be able to develop systems that actually learn, and are not just programmed (Domain Specific Langauges hint at just the beginning of our trip down that path).

The preceding blog entry has been syndicated from the DonXML Demsak’s All Things Techie Blog. Please post all comments on the original post.

Not the Way to Introduce XmlTextReader

DonXML — Tue, 30 Nov 2004 02:59:00 GMT

Thom Robbins is a great guy, but unfortunately for him he has bumped into one of my major pet peeves, Viral Coding Examples with his Introducing the XmlTextReader post. It really isn’t his fault, since the code he uses is very similar to the code example in the XmlTextReader.Read() documentation, and I complained about that code to the System.Xml team at the MVP Summit. I did promise to write something up on it, and Thom’s post finally got me to do it (it has been months since I promised to write this up).

The problem is in the structure of the code:

Dim xmlFileStream As New FileStream("cust.xml", FileMode.Open)
Dim xmlRead As New XmlTextReader(xmlFileStream)

While xmlRead.Read
    xmlRead.MoveToContent()
     If xmlRead.HasValue Then
         MsgBox(xmlRead.Value)
     End If
End While

xmlRead.Close()

xmlFileStream.Close()

At first glance the code looks perfectly fine. But, knowing full well that some developer new to System.Xml will be using this code as a template for bigger things we have an obligation to make it easier for them to adapt this code without causing “strange” errors.

Problem #1

No explicit setting of the WhitespaceHandling option. Unless the developer is familiar with XmlTextReader (which shouldn't be in this case), they would not know that the default is WhitespaceHandling.All, which causes the reader to return all Whitespace and SignificantWhitespace nodes (which will definitely confuse the developer). So after the declaration of the xmlRead variable you should set the WhitespaceHandling property.

xmlRead.WhitespaceHandling = WhitespaceHandling.None

At least now the developer realizes that there is a property for WhitespaceHandling, and will/can change it as needed.

Problem #2

Implicit Control of Reads in While Loops. My biggest problem with the code examples used to XmlTextReader has to do with the While xmlRead.Read loops. Although it looks very harmless, the while loop that executes a read at the beginning (or the end) of a looping structure will cause bugs to creep into the code because of the way other methods on the XmlTextReader handle the cursor used to point to the current node. If all you do is execute Reads via the while loop, you are perfectly fine. But once you add code that manipulates the cursor from within the while loop, now you run the chance of skipping nodes accidentally.

Here’s a great example. You have an XML stream that looks like this:

<ROOT>
<LEVEL1>
<LEVEL2>1st Level2 text node</LEVEL2>
<LEVEL2>2nd Level2 text node </LEVEL2>
</LEVEL1>
</ROOT>

And you want to print out the contents of the elements level2, so you modify the standard code example to look like this:

While xmlRead.Read
    xmlRead.MoveToContent()
    If xmlRead.IsStartElement() then
        If xmlReader.Name = “level2” then
           MsgBox(xmlRead.ReadInnerXml())
        End If
    End If
End While

And you know what, it works fine. But say the XML stream does not have all that pretty whitespace, or that they took my advice and set the WhitespaceHandling property (in this case to None). Now the code doesn’t work, since you had a bug in your code and you didn’t know it. What? How is that? Well, the ReadInnerXml method reads to the first node past the EndElement. In the case of the XML Stream with the nice formatting (and when WhitespaceHandling is All) the next node is a whitespace node, and when the while loop fires the Read method, all is well and the cursor is moved to the next Node (which should be the next StartElement, otherwise the MoveToContent method will move you to the next content node (which is any node that is non-white space text, CDATA, Element, EndElement, EntityReference, or EndEntity)). But without the whitespace nodes to stop the ReadInnerXml method the cursor is positioned at the next non whitespace node (which in this case is the StartElement for level2) and then the while loop fires the Read method, and when we enter the loop we have now read past the StartElement and the if condition is not met (and we skip the whole element).

So, what is a better example for an introduction to the XmlTextReader? Explicitly control when a Read is executed.

XmlRead.WhitespaceHandling = WhitespaceHandling.None
Dim Continue as Boolean
If xmlRead.Read = False then
    Continue = False
End If
While Continue
    If xmlRead.IsStartElement then
        If xmlRead.Name = “level2” then
            MsgBox(xmlRead.ReadInnerXml())
        Else
            Continue = xmlRead.Read()
        End If
    Else
        Continue = xmlRead.Read()
    End If
End While

Now we have explicit control over when a Read is executed, and in the case of rogue methods that place your cursor to the next node (that you haven’t tested yet), you can skip the implicit Read.

If you want, you can download a fully functional example with 5 different test cases.

The preceding blog entry has been syndicated from the DonXML Demsak’s All Things Techie Blog. Please post all comments on the original post.

Viral Coding Examples in Presentations

DonXML — Thu, 21 Oct 2004 21:49:00 GMT

Joe Fawcett (fellow XML MVP) came across a great example (from the Microsoft.Public.Xml newsgroup) of one of my biggest pet peeves, “We (the community) are doing a very poor job teaching the average developer how to use XML properly in .Net”.

I want to draw your attention to a line from the original post:

“So, is it possible to directly modify the xml file instead of using the dataset.”

And the first response was:

“you can do it using Data Island”

Why does thing question bug me so much? Because we (the community) have done a very bad job using XML correctly in our articles and presentations. The reason why Joe pinged me with this example is because I went on a rant earlier this year (in a private conversation) how disappointed I was in one of the DevDays presentations. In the presentation, they were trying to show some cool Whidbey WinForm stuff, and the example went something like this: “We have this XML file and we want to display the info it in a WinForm, so let’s use a DataSet to open the XML file and then bind it to the controls on the form”. NOOOOO my mind went. This is not the way to introduce developers to XML. Yes, can you open an XML with a dataset, and yes it is quick and easy to do, but it is enforcing very bad development practices, which lead to questions like Joe found. People learn new technologies from these examples, and we really have to be careful when writing them.

The preceding blog entry has been syndicated from the DonXML Demsak’s All Things Techie Blog. Please post all comments on the original post.

NJ Dev Dinner 9/14/2004 – Special Edition

DonXML — Mon, 13 Sep 2004 18:31:00 GMT

Right after the N3UG meeting (with Dave Douglass’ presentation on Generics), we will have a special edition of the NJ Dev Dinner. What’s the occasion? Microsoft Developer Evangelist Kirk Allen Evans is coming up from Atlanta, GA, and will be in the area. After the meeting we will move on over to Chevy’s for some fun, food, and drink.

The preceding blog entry has been syndicated from the DonXML Demsak’s All Things Techie Blog. Please post all comments on the original post.

A Suggestion for the Microsoft Careers Site

DonXML — Wed, 01 Sep 2004 14:43:00 GMT

I was on the Microsoft Technical Recruiting Chat last night (I’m not really looking, just snooping around), when I remembered a comment made by either Zoë or Gretchen (I can’t remember who, or even find it) about making sure that your resume looked good in plain text when submitting it via their site. They don’t take resumes in Word format (or even HTML), and you can either paste vanilla text into a free form input box, or use the resume builder. They mentioned that as an applicant, you should make sure the plain text version of your resume is easy to read, and not just a Word doc saved as text (which would be very hard on the eyes). At the time I read that, I thought it was a good tidbit, so I filed it away for future reference. During the chat something clicked, and I remembered that Microsoft had released an Office Solution Accelerator for Recruiting, which includes a nice resume builder section. The resume builder includes an InfoPath Resume Form (which uses the HR-XML schema for resumes), and a complementing resume submittal web service. Shouldn’t Microsoft recruiting IT department do a little dogfooding, and be using this? It is definitely better than submitting a text version of a resume, and trying to shred the document. Plus, since it is an XML document, they could easily apply an XSL (either XSLT to WordML or XSL-FO to PDF) and transform the resume into an easy to read format for the recruiters. I know that everyone does not have InfoPath (but it is an excellent tool), so they would have to create an alternate means to creating an XML version of the resume (can we say open source project?), and still maintain the good old text box input method for the less technical folks applying to Microsoft. It sounds to me like there would be a decent return on investment for this type of thing, with Microsoft hiring some 7000 people this year.

The preceding blog entry has been syndicated from the DonXML Demsak’s All Things Techie Blog. Please post all comments on the original post.

DataMapper Performance Statistics On VS2005 Beta 1 and SQL Server 2005 Beta2

DonXML — Mon, 09 Aug 2004 15:15:00 GMT

I took the code from my previous DataMapper examples, which was written for .Net 1.1 and SQL2K, and ported it to VS2005 Beta 1 with SQL Server 2K5. I did not have to change one line of code in either the sprocs or in the 1.1 C# code (great going VS & SQL team). Since my example code also used the DAAB 3.1, I decided to port that too, with no issues. Since I was going to be testing the beta versions running in VPC, I new that I would not be able to compare my original stats to the new stats. But since what I’m really after was performance differences versus a baseline of the DataReader, and comparing those numbers, running on a host OS versus a VPC OS should not be a factor (as long as the OS is the same, which it was, Windows XP).

So let’s look at the numbers.

The original stats (using .Net 1.1)

DataReader	341
DataSet	411	21%
Xpath	450	32%
XmlSerialization	542	59%

Using .Net 2.0 beta 1 with SQL Server 2K5 Beta 2

DataReader	1545
DataSet	1772	15%
Xpath	1993	29%
XmlSerialization	2069	34%

So, when comparing to a 2.0 base of DataReader, the other 3 techniques all picked up their performance (relative to the 1.1 ratios). The biggest gain was in XmlSerialization performance which cut the ratio in half (which makes sense since most of hard lifting is done in the generated code). Now I need to get down and dirty and do 2 things, tweak the .Net code to use 2.0 features (like generics and the new XPathNav) and then tweak the SQL Server sprocs to take advantage of the better SQLXML features.

But overall, considering this is still beta code and not totally performance tuned, these are incredible numbers.

The preceding blog entry has been syndicated from the DonXML Demsak’s All Things Techie Blog. Please post all comments on the original post.

Revisiting XML DevCon 2003 – The Videos

DonXML — Wed, 14 Jul 2004 15:49:00 GMT

It is hard to believe, but it has been just over one year since XML DevCon 2003. Because most of the subject mater was pretty ahead of its time for the common enterprise developer, I’m sure most of the people that read this blog either didn’t watch the videos on MSDN TV, or only understood a limited amount of the content. Now with another years worth of experience under your belt, you should take the time to watch them again. I’ve found lots of little tid bits that I missed the last time.

Don Box -- SOAP: Service Oriented Architecture and Programming, part 1, part 2
Tim Ewald -- Rebuilding MSDN with Angle Brackets (real world experiences of XML Schema), part 1, part 2
Aaron Skonnard -- Validating Business Rules with XPath Assertions, part 1, part 2

The preceding blog entry has been syndicated from the DonXML Demsak’s All Things Techie Blog. Please post all comments on the original post.

Performance Statistics of Various Implementations of the Data Mapper Pattern

DonXML — Mon, 28 Jun 2004 13:01:00 GMT

What started off as a quick how-to example of rehydrating business objects from data access layers (in reply to this Jay Kimble post) morph’d into a whole lot more (thanks to Scott Hansleman’s DataSet post, and my thoughts on the topic). The idea was pretty simple. I wanted to see how long it took to rehydrate a multi-dimension object graph from data retrieved from a database using the most common methods in .Net. Going into this project, I had some preconceived notions on what the results would be, but I was very surprised at the results.

For those of you that are not pattern aware (yet), the process that you go through when converting data to business objects is called a Data Mapper Pattern. The Data Mapper layer handles the creation of your business objects (aka your domain) from data, and hopefully don’t even know that there is a database (thanks to the data access layer). In a domain-less (aka no business object) world the mapping step is skipped entirely, and datasets are used, with the business rules all tied up in the presentation tier or down in the database. A slight advancement over the pure dataset method is strongly-type datasets, but unless you modify the generated code, you can not add the business rules into your strongly-typed datasets (at least until .Net 2.0’s partial classes).

In this project, I decided to use the DAAB v3.1 as the basis for my data access layer, since I know that a lot of people use it, and it saves me from have to explain my data access layer (more on that another time). All I did is add my Connection String encryption project to it (keeping it a more enterprise ready example), and added a service layer around the DAAB (to encapsulate the Data Access, and hide it as much as possible from the business objects).

For the actual data mapping, that code is actually only dependent on one thing, how you exposing the data to the data mapper code. There are three popular ways to pass data from the data access layer to the layers above, the DataReader, the DataSet, and the XmlReader. Both the DataReader and the DataSet require you to create the actual data mapping code by hand, but with the XmlReader, you can use Xml Serialization to deserialize the XML into an instance of your business objects. There is one more way that I’ve been working on to pass data up from the data access layer, and that is by using XPathNavigators, so I added this as a fourth example. This is the one implementation that I was most interested in finding out the performance figures for.

The sample database I used was the all too familiar Northwind database on SQL Server 2000 (with all the latest service packs). I created a cloned copy of the database and created the following stored procedure to test against (well actually there are 2 versions of the stored proc, one “normal” proc, and one that returned an XML stream using the For Xml Auto clause) that would return all the employees, with their associated Territories, and the Region for the Territory.

SELECT Employee.EmployeeId as "Employee.Id",
Employee.LastName as "Employee.LastName",
Employee.FirstName as "Employee.FirstName",
Employee.Title as "Employee.Title",
Employee.BirthDate as "Employee.BirthDate",
Territory.TerritoryId as "Territory.Id",
RTRIM(Territory.TerritoryDescription) as "Territory.Description",
Region.RegionID as "Region.Id",
RTRIM(Region.RegionDescription) as "Region.Description"
from dbo.Employees Employee
INNER JOIN dbo.EmployeeTerritories et
on Employee.EmployeeID = et.EmployeeID
INNER JOIN dbo.Territories Territory
on et.TerritoryID = Territory.TerritoryID
INNER Join dbo.Region Region
on Region.RegionID = Territory.RegionID
order by Employee.EmployeeID, Territory.TerritoryId, Region.RegionId;

This would give me a nice 3 dimensional object graph to test the performance numbers against. If you want to see all the code, you can pull it down from here (Or you can get the code from the Mvp.Xml SourceForge Project). Just add the stored procs to a version of Northwind, and compile and walk thru the code (until I can get some time to write this all up in a series of articles).

Now, on to the results. I used my Dell 8500 laptop as the test machine, with WindowsXP SP2, VS.Net 2k3, and SQL2k. I wrote a small console app for each of the 4 test cases and they all used the High Performance Timer code to get better time measurements. In order to try to get more consistent results, the console app will load the business object one time (to get everything compiled and into memory, including connection and all the SQL Server side stuff), and then start the timer and loop thru 1000 times.

DataReader - 341

DataSet - 411 ( 18.6% slower than the DataReader)

XPathNavigator - 450 (27.7% slower than the DataReader)

XmlSerialization - 542 (46.3% slower than the DataReader)

It wasn’t real surprising that the DataReader was the fastest of the implementations, but what surprised me was how slow the XmlSerialization was, and that the XPathNavigator was slower than the DataSet. (oh, and if you are wondering how I calculated the percent changed, see this article on why what you were taught in high school was wrong and very misleading). I went over my code pretty well, but I’m sure that people will find some performance enhancements, but overall, I’m pretty sure that things were done pretty consistently, and most enhancements will effect all the results and wash out of the final analysis ( and yes, I made sure that I cached the XmlSerialization instance, so that isn’t why that implementation method is so slow). But, if you find something, definitely let me know so that I can update it.

I’m sure there are lots of other ways to implement a DataMapper in .Net. You are welcome to clone the code, implement it, and let us know what your results were.

The preceding blog entry has been syndicated from the DonXML Demsak’s All Things Techie Blog. Please post all comments on the original post.