Mehfuz's WebLog

REST to collection builder using LINQ to XML

Few weeks ago, I did a post about how can I handle REST response using LINQ To XML. You can take a look at it here.

http://weblogs.asp.net/mehfuzh/archive/2008/01/11/rest-with-linq-to-xml.aspx

Now , this is nice that we can parse REST response with LINQ To XML but at the same time it is kind of boring writing the mapping manually every time. In my Linq.Flckr project, I need to build objects from REST in various occasions, so its kind of monotonous as well as error prone writing the XML to Object Mapping manually, even its really easy to do using LINQ to XML. Therefore, I have come up with a tiny class that does the task for me and in return, all I have to do is to declare some attributes on top of my Class and Property.

Once, I have added all the property mappings, all is left is to create the RestCollectionBuilder object and call the ToCollecton to get the IEnumreable<T> result from REST response.

RestToCollectionBuilder<SomeObject> builder = 
new RestToCollectionBuilder<SomeObject>();
OR
// pass the root element from which attributes 
//will be passed to the SomeObject class.
RestToCollectionBuilder<SomeObject> builder =
 new RestToCollectionBuilder<SomeObject>(##ROOELEMENT##);
 // Where T: IDisposable
// Build the object.
IEnumerable<SomeObject> list = builder.ToCollection(##REQUESTURL##);
OR
// get the element and do some parse of your own, if needed.
XElement element = GetElement(..) 
IEnumerable<SomeObject> list = builder.ToCollection(element);

Talking about attributes and elements , let's consider the following REST response from Flickr .

<rsp stat="ok">
  <person id="12037949754@N01" nsid="12037949754@N01"
 isadmin="1" ispro="1" iconserver="122" iconfarm="1"
 gender="M" ignored="0" contact="0" 
friend="0" family="0" revcontact="0"
 revfriend="0" revfamily="0">
    <username>bees</username>
    <realname>Cal! Henderson@</realname>
    <mbox_sha1sum>2971b1c2fd1d4f0e8f99c167cd85d522a614b07b</mbox_sha1sum>
    <location>San Francisco, USA</location>
    <photosurl>http://www.flickr.com/photos/bees/</photosurl>
    <profileurl>http://www.flickr.com/people/bees/</profileurl>
    <mobileurl>http://m.flickr.com/photostream.gne?id=287</mobileurl>
  </person> 
</rsp>

Here , we need to tell the Collection builder which element is mapped to which property.Accordingly, I have created some custom attributes that will be used by the builder to create the map. Which are

• XElementAttribute
• XAttributeAttribute
• XNameAttribtute

The first two attributes inherits the XNameAttribute. So, if our class is People and we need to map the person element and its descendants to it, our class will look like

// maps the element to which the class belongs to
[XElement("person")] 
public class Person
{
    // maps to <person nsId ="xx">
    [XAttribute("nsid")]
    public string Id { get; set; }
   // maps to <person ispro ="True">
    [XAttribute("ispro")]
    public bool IsPro { get; set; }
    // maps to <username>Henderson</username>
    [XElement("username")]
    public string Username { get; set; }
   
   ....
   ....
   ....
}

And that's all is needed to map the class to REST elements though under the hood, RestToCollectionBuilder class does use LINQ To XML to prepare the collection, but gives out a more strong solution to work with REST responses. I have used this technique in Linq.Flickr. You can check that out by digging in the code. Along with that, I have fused in a tiny Console App that processes the REST response, builds the object and dumps the result to the screen, you can download it here as well.

LinqExtender 1.3.1

Last week, I have released a patch for LinqExtender project at Codeplex. You can find the full feature list at release page. One issue that I will talk about here is orderby clause and its related in-memory sort.

Service based apps are of fixed set of operations. Now, it might happen that you need the list of most of popular photo tags from Flickr and you need to sort them by title as well. But, unfortunately you come to know that it has no parameter that sorts by title.

Now , those who have played around the toolkit (since there are lots of downloads :-)) , must know that for the following query

var query = from tag in _context.PopularTags
            where tag.Period == TagPeriod.Week orderby tag.Title ascending
            select tag;

I will have a Bucket.OrderByClause inside Query<T>.Process override. Bucket.OrderByClause has two properties (FieldName, IsAscending)  by which, we can understand the orderby query nature and take action on that. This is good , if external method that we are calling supports the orderby info. If not , then alternatively we can do something that follows

protected override void Process
(LinqExtender.Interface.IModify<T> items, Bucket bucket)
{
    IEnumerable<QueryObject> results = GetResult( ... );    
   // add the result to the extender collection, "true" defines that it should be 
   // be sorted by the orderby specified in query expression.    
    items.AddRange(result, true);         
}

OR

protected override void Process
(LinqExtender.Interface.IModify<T> items, Bucket bucket)
{
    foreach (var item in unfilterItems) 
    { 
       // do some of work your own work
       // finally add it to the collection
       items.Add(item);
    }
    // sort the items if any orderby query is used 
   items.Sort();
}

this.Sort(new QueryItemComparer<T>
(_queryObject.OrderByClause.FieldName, _queryObject.OrderByClause.IsAscending));

Project url : www.codeplex.com/linqextender

OpenXML to parse your office documents

ECMA OpenXml is a recognized  open standard for saving and retrieving  office documents that enables cross-platform document porting and sharing. The Office 2007 uses this format for its data persistence for word, excel and power point lineups. There is a OpenXml SDK CTP available for it to download from MSDN, which lets you create your own office component that works on universal format. 

Now, using OpenXml SDK creating office components is easier than before, also it promises to bring you cross product and platform flavor.

OpenXml document generally looks like

<w:document
 w:xmlns="http://schemas.openxmlformats.org/wordprocessingml/2006/main">
  <w:body>
     <w:p>
        // items goes here
     </w:p>
  </w:body>
</w:document>

<w:p> wraps up every paragraph and below it goes all the style elements and text nodes.  Now, the reason why I mentioned this OpenXml here is LINQ.  In a moment, I will show how it is possible to create an easy word document parser using the OpenXml SDK and a bit LINQ.

Now starting , you have to add the following reference to your project.

This Dll comes a part of the OpenXml SDK , you can either copy it to your project or ref it from where it is installed, it is not installed in GAC. So, I supplied it with the download provided with this post as well.

This is the sample document that we will be parsing using LINQ and OpenXml SDK.The main thing to do so, is to create the processing document, which takes a file path / stream and a bool value named readWriteMode, true means both way.

using (WordprocessingDocument doc = WordprocessingDocument.Open(_path, true))
 {
         MainDocumentPart mPart = doc.MainDocumentPart;

          using (StreamReader reader = new StreamReader(mPart.GetStream()))
          {
                
          }
}

Now, the starting node for the processing doc is MainDocumentPart, which is divided up into several OpenXmlPart derived objects (base of all the document parts), we can work with the whole document or with smaller parts basis on our data need. Anyway, next and the only thing is to get around a stream for the XML doc and process it with LINQToXML.

XDocument xDocument = XDocument.Load(XmlReader.Create(reader));

So , the step is to use XmlReader.Create to get a clean XML and then pass it to XDocument , as there are special characters in the stream, which the XDocument cant process directly.
We also need to create XNameSpace and XName elements, which will be used to query the document for what we are looking.

XNamespace w = "http://schemas.openxmlformats.org/wordprocessingml/2006/main";
// the elements we will be looking for data.
XName rPr = w + "pPr";
XName p = w + "p";

Finally, its all LINQ to get the list of text blocks and styles attached to them , in case of this document there are three blocks (1. Title 2. br 3. Text).  The parsing and fill up looks like the following, here a lot of null checks are used to avoid pitfalls ,  as the nodes are not consistent all the way down.

var query = from element in xDocument.Descendants(p)
            select new Document
            {
                ItemProperty = element.Element(rPr) != null ?
               ((from sElement in element.Descendants(rPr)
                 select new ItemProperty
                 {
                     Style = sElement.IsEmpty == false ?
                     (sElement.Element(w + "pStyle") != null ?
 sElement.Element(w + "pStyle").Attribute(w + "val").Value : 
string.Empty) : string.Empty,
                     Lang = sElement.IsEmpty == false ?
                     (sElement.Element(w + "lang") != null ? 
(sElement.Element(w + "lang").Value ?? string.Empty) : 
string.Empty) : string.Empty
                 }).First<ItemProperty>()) : null,
                Text = element.Value == string.Empty ? "<br/>" 
: element.Value

            };

return query.ToList<Document>();

In the code, Document is the custom class that looks like

public class ItemProperty
{
    public string Style { get; set; }
    public string Lang { get; set; }
}
public class Document
{
    public string Text { get; set; }

    private ItemProperty _itemProperty = new ItemProperty();

    public ItemProperty ItemProperty { get; set; }
}

That's it , we got the document in the memory , now either we can print it in console or make our custom viewer to show it, but for the time being I will print the lines on console :-)

// the function whose code is shown above
IList<Document> list = GetParagraphs();

foreach (Document doc in list)
{
    Console.WriteLine(doc.ItemProperty.Style + ":" + doc.Text);
}

Download the full source here

Have Fun!!

A video article site initiative by dotnetvideos

Few months back Ravi stated a video site http://www.dotnetvideos.net/  at his own initiative. I did registered and today I got a mail from http://www.aspnetpro.com/ that I have got a free 6 month premium account. The concept is that early signup users will be credited with free subscription of aspnetpro account. Though, there are plenty of places to improve but as an singleton initiative, dotnetvideos looks good. I heard from Ravi also that they will be offering a book every week from Apress.com beginning April 2008 for one random user who regularly visits the site.

One, thing I did noticed that aspnetpro gives out their latest magazine issue for free so its a nice thing to check out as well, the content collection looks pretty nice.

IE8 from MIX08 to developers

Although I was not in mix08, I took interest in knowing every detail of how it is going on and downloaded the sessions and keynote as it is available at http://sesssions.visitmix.com , a clean Silverlight site with all the cool videos. This morning , I was watching out the keynote, and found Dean Hachamovitch (General manager,IE)  point out some of the cool new features that really dropped my jaws.

Apart with the new W3C compliance and CSS 2.1 along with HTML 5, two things that really amazed me are (which I will be describing shortly)

1. Ajax Navigation
2. Connectivity Events.

Firstly, in order to use all the IE8 and HTML 5 features, we need to declare a doc type on top of our HTML file that looks like

<!DOCTYPE html>

This is the doc type of HTML 5, without using this doc type it is also possible to test your IE8 compatible script and for that SHIFT + F12 hotkey should be pressed, which will bring up the "Developer Tools" console, there from the View menu , you have to specify the compatibility mode to IE8 and hit F5 on your browser to see the effect :-)

Now, Let's start with simple but cool one called "Connectivity Events". It often happens that I am writing a big comment in someone's blog by the time ISP goes down or page expired, and to my frustration I write it again anyhow. But in IE8 you can let users save their data in case of anomalies by using the window.navigator.onLine property.Using this,  you can check out if you lost your connectivity with the server and show your user with proper message, rather make him redo things again.This is pretty simple and I fused out a demo script which does right that.

I declared a DIV block that contains the message , when there is no connection.

<div id="offlineDiv" style="color:Red;font-size:14pt;display:none;">
Connection Status: offline
</div>

And another DIV block , that is visible when you are online.

<div id="onlineDiv" style="color:Green;font-size:14pt;display:block;">
Connection Status: online
</div>

And I wrote a sample script that uses the window.navigator.onLine , to check it out and put it in the head tag. The JS actually toggles the div on and off depending on the connection status.

function check()
{
 var online = window.navigator.onLine;
 
 if (!online) 
 {
   document.getElementById('offlineDiv').style.display = "block";
   document.getElementById('onlineDiv').style.display = "none";
 }
 else
 {
   document.getElementById('offlineDiv').style.display = "none";
   document.getElementById('onlineDiv').style.display = "block";
 }
}

Now, either I can execute this JS in body onload event or at the end of the function  or I can further use the ononline  and onoffline event in the body tag. Either way it will produce the same result.

<body ononline="check()" onoffline="check()">

...

</body>

You can see the online/offline status by checking/unchecking the tools->work offline or plugging on/off your Internet :-)

Moving to the another feature called  "Ajax Navigation". We all know that browser back does not work properly with Ajax apps, you can read it further at MSDN, but I will do show the pointer to use it for making your app taking the advantage of it. The magic is done by a simple but a mighty window.location.hash property. Now, it is possible to set hash for pre IE8 browsers as well, but what makes it special is that the browser creates a history entry for the changed hash URL by which we can go back to the wizard step or map location by pressing the browser back in our favorite Ajax app.

But just going back and forth does not solve the problem , as we need to reproduce the step depending on the hash value and for that we need to trigger the appropriate function , when hash is changed. IE8 takes it further by introducing a window event called onhashchange.

window.onhashchange = function() { // your code goes here; }

This can be placed anywhere in the "Head" of your html doc. I have created a script file with step 1, 2 and 3 using some DIV and text, go to different steps and press browser back to see the effect, both this and the other script is added here (To Download) for you.

Finally, to try these and many more  grab a copy of IE8 Beta from MSDN download center and make your app ready for IE8 today :-).

Updated on March 19, 2008

Moving to Telerik

I started working for Pageflakes around 2 years from now, it was not only all the way fun to work as an early starter but being a core member of a startup is really a lot to learn.

As of now, I am moving  to Telerik Inc , to play out with some cool new technologies and products that they have made out and make something even interesting in coming days. Again, it is really great working with Pageflakes team and it will surely be the #1 and all my regards go with it.

[Update : Pageflakes is acquired by LiveUniverse ]