Tales from the Evil Empire

Bertrand Le Roy's blog

News


Bertrand Le Roy

BoudinFatal's Gamercard

Tales from the Evil Empire - Blogged

Blogs I read

My other stuff

Archives

Clay: malleable C# dynamic objects – part 2

(c) Bertrand Le RoyIn the first part of this post, I explained what requirements we have for the view models in Orchard and why we think dynamic is a good fit for such an object model.

This time, we’re going to look at LouisClay library and how you can use it to create object graphs and consume them.

But before we do that, I want to address a couple of questions.

1. If we use dynamic, aren’t we losing IntelliSense and compile-time checking and all the good things that come with statically-typed languages? And is C# becoming overloaded with concepts, and trying to be good at everything but becoming good at nothing?

Hush, hush, everything is going to be all right. Relax.

Now think of all the XML/DOM styles of APIs that you know in .NET (or Java for that matter). Except if they are doing code generation, you already don’t get IntelliSense and compile-time checking. Well, you do get them on the meta-model, which you care very little about, but not on the actual data, which you do care a lot about. So it’s ok, we can afford to lose what we didn’t have in the first place. And there’s always testing, right?

As for the evolution of C#, come on. You should be happy it’s still alive and innovating. Take a deep dive into what expression trees really mean for example. It’s a beautiful thing.

2. What’s wrong with ExpandoObject?

Nothing but we can do better. ExpandoObject is actually implemented in a surprising way that makes it very efficient. Hint: no dictionary in there. Other hint: it’s a beautiful thing.

But in terms of API usability it’s not very daring and in particular it does not do much to help you build deep dynamic object graphs. Its behavior is also fixed and can’t be extended.

Clay on the other hand is highly extensible and focuses on creation and consumption of deep graphs.

Let’s get started. The first thing you can do with Clay is create a simple object and set properties on it. Before we do that, we’ll instantiate a factory that will give us some nice syntactic semantic sugar. I wish we could skip this step and use some kind of static API instead but we can’t. Well, small price to pay as you’ll see:

dynamic New = new ClayFactory();

Now this “New” object will help us create new Clay objects, as the name implies (although this name is just a convention).

Here’s something easy and not too surprising:

var person = New.Person();
person.FirstName = "Louis";
person.LastName = "Dejardin";

Nothing you couldn’t do with ExpandoObject but where this gets interesting is that there is more than one way to skin that cat, and that is going to open up a lot of possibilities.

For instance in Clay, indexer syntax and property accessors are equivalent, just as they are in JavaScript. This is very useful when you are writing code that accesses a property by name without knowing that name at compile-time:

var person = New.Person();
person["FirstName"] = "Louis";
person["LastName"] = "Dejardin";

But that’s not all. You can also use properties as chainable setters, jQuery-style:

var person = New.Person()
    .FirstName("Louis")
    .LastName("Dejardin");

Or you can pass an anonymous object in if that’s your fancy:

var person = New.Person(new {
    FirstName = "Louis",
    LastName = "Dejardin"
});

Even better, Clay also understands named arguments, which enables us to write this:

var person = New.Person(
    FirstName: "Louis",
    LastName: "Dejardin"
);

In summary, there is a lot of ways you can set properties and initialize Clay objects.

As you’d expect, accessing properties can also be done in a number of ways and all of the following are equivalent:

person.FirstName
person["FirstName"]
person.FirstName()

You can also create JavaScript-style arrays:

var people = New.Array(
    New.Person().FirstName("Louis").LastName("Dejardin"),
    New.Person().FirstName("Bertrand").LastName("Le Roy")
);

That array is also a full Clay object, meaning that you can add properties to it on the fly.

Then you can do this in order to count the items in the array and to access the FirstName property of the first item in the array:

people.Count
people[0].FirstName

It’s even easier when you want to create an array property on an existing Clay object:

person.Aliases("bleroy", "BoudinFatal");

If more than one argument gets passed in, Clay is assuming that you’re initializing the property as an array. And if you only have zero or one arguments, just explicitly pass in an array (CLR or Clay):

person.Aliases(new[] {"Lou"});

Contrary to CLR arrays, Clay arrays can grow dynamically:

person.Aliases.Add("loudej");

And they also answer to a number of methods such as AddRange, Insert, Remove, RemoveAt, Contains, IndexOf, or CopyTo.

Getting all this together, we can create a reasonably complex object graph with a fairly expressive and terse syntax:

var directory = New.Array(
    New.Person(
        FirstName: "Louis",
        LastName: "Dejardin",
        Aliases: new[] { "Lou" }
    ),
    New.Person(
        FirstName: "Bertrand",
        LastName: "Le Roy"
    ).Aliases("bleroy", "boudin"),
    New.Person(
        FirstName: "Renaud",
        LastName: "Paquay"
    ).Aliases("Your Scruminess", "Chef")
).Name("Some Orchard folks");

There’s one last thing I’d like to show that I found really neat and surprising the first time Louis showed it to me.

Imagine that you have a CLR interface that you need to implement, for example:

public interface IPerson {
    string FirstName { get; set; }
    string LastName { get; set; }
}

but you would like to do it using a Clay object such as one of the persons defined above. Well, you can do this:

IPerson lou = people[0];
var fullName = lou.FirstName + " " + lou.LastName;

What’s extraordinary here is that lou is a perfectly valid statically typed CLR variable. You’ll get full IntelliSense and compile-time checks on that code. It’s just an object that implements IPerson although we never wrote a concrete type implementing that interface.

What makes the magic possible is that Clay is overriding the cast operator and creating a dynamic proxy for the interface (using Castle) that delegates the members to the Clay object.

So there is an actual CLR type but it’s being code-generated on the fly.

That is what enables you to write the following:

foreach(var person in directory) {
    Trace.Write(person.FirstName);
}

What’s happening here is that the “directory” Clay array is being cast into an IEnumerable, for which all the right methods are implemented by the dynamic Clay array object.

We are going to dig deeper into Clay in future posts as there is so much more to do and hack but that should give a satisfactory introduction of the basic intents for the library. I certainly hope you like it and find some crazy ideas about how to use it.

You can find Clay here: http://clay.codeplex.com/

The first part of this post is here:
http://weblogs.asp.net/bleroy/archive/2010/08/16/clay-malleable-c-dynamic-objects-part-1-why-we-need-it.aspx

Comments

InfinitiesLoop said:

The implicit casting (coin term: molding?) is brilliant and would seem useful for other scenarios too. For example, you've got an IDictionary<String,Foo> and you want to treat it as an interface that defines some properties that happen to be keys in the dictionary.

ISomething something = Clay.Proxy(dictionary);

something.Value1 = 1; // dictionary["Value1"] = 1;

For gets, it could also implicitly 'mold' the value, for cases where the value itself is also a dictionary:

Foo foo = something.PropertyOfTypeFoo;

// Clay.Proxy(typeof(Foo), dictionary["PropertyOfTypeFoo"])

This would be great for JSON deserialization from JavaScriptSerializer, where you've got a hierarchy of objects as dictionaries that all boil down to just simple values in the leaves. Currently you pretty much have to manually build the objects from the Dictionary.

# August 18, 2010 1:58 PM

srulyt said:

Awesome!

It would be cool if Clay would also allow access events that could be set like in jQuery.

so Person.FirstName() would return the first name and Person.FirstName(x => LogAccess(x));

or whatever other function you may want to hook in. This could allow the setter of the data to hook in and see who accessed it. Could be very useful in a CMS type environment where you may have multiple modules adding data and multiple partial views accessing it.

Besides all ofthis. How is performance?

# August 18, 2010 2:12 PM

yysun said:

This is cool!

My question is if Clay object supports data binding?

# August 18, 2010 3:26 PM

Bertrand Le Roy said:

@InfinitiesLoop: yep, that would be an interesting application. Actually, serialization/deserialization is something that would be interesting to add.

@srulyt: well, the good news is that this would be fairly easy to add as an extension. More on that in future posts. The bad news is that assigning a Lambda to a dynamic member is not as easy as it sounds: the C# compiler doesn't let you do it without some expression goo.

As for performance, we haven't looked at it yet, and it's using dictionaries internally but in principle it could use the same tricks that ExpandoObject is using and get quite fast. Anyway, for the moment we want to use that where performance is not a big issue and where convenience is way more important.

@yysun: not yet. What exactly do you have in mind?

# August 18, 2010 4:11 PM

Robin Debreuil said:

wait, I do get intellisense/serialization/compile time errors etc when I make classes for data...? Why is that the wrong way again?

# August 18, 2010 5:40 PM

Bertrand Le Roy said:

@Robin: it's not "the wrong way" if it works for what you're doing. But you need to create a new class for each shape and more importantly you can't graft new members after the fact. With this, you can have a multitude of agents contributing to building the same object graph and adding stuff that wasn't planned for initially.

# August 18, 2010 5:46 PM

Vincent said:

This is amazing. Just tested it out and it feels very smooth.

The only thing is that you lose Intellisense when you pass the created object graph into a View of an MVC app (ViewPage<dynamic>).

Still doing <%= Model.FirstName %> perfectly renders the correct value of course.

Or is there a way to "enforce" Intellisense for a dynamic type?

# August 19, 2010 1:33 AM

Robin Debreuil said:

I'm pretty sure you would agree 'less typing' isn't a good reason to give up compile time safety, but I've done a ton of AS/JS and understand the attraction

I've never really worked in a domain where I needed to munge unknown types together at runtime though. REALLY not questioning it, I know everyone has a different set of problems in different domains - I am curious what kind of things that is though..? Is that like user generated data? This is OT, feel free to ignore, but isn't handling data like that basically impossible anyway? Or is it more wrap it and send along, or just display it? Or maybe you get code that handles the objects from the same source?

Always cool to look at problems through the eyes of others : ).

To be honest, I do cringe a little seeing C# able to run blind like JS, but if you are authoring it I'm sure it is sensible and in good hands : ).

# August 19, 2010 3:01 AM

Bertrand Le Roy said:

@Vincent: as the article shows, you can cast into an interface type and get IntelliSense but if are willing to do that, it's actually quite likely that you will get little benefit from not using a statically-typed object as your model.

@Robin: community-created modules can all chime in on the creation and enrichment of any node in the tree of view data that can be displayed on an Orchard page by community-created themes. Even a specific node might get enriched with new properties and children by multiple agents. You've got tiny bits of data being collaboratively composed into the view model, and tiny bits of themed templates rendering them.

What you trivialize by referring to it by "less typing" is actually quite important. In particular when dealing with a designer audience, being able to write item.title rather than item.Attributes["title"].Value is very important. And as I said in the article, you *already don't* get IntelliSense on what matters here, the data.

It shouldn't be that surprising that people prefer to manipulate an HTML DOM using jQuery rather than the native DOM API. One is fluent, dynamic and concretely data-centric while the other is static and focuses on the abstract structure of the tree.

Same thing here. I'm certainly not advocating that you use Clay where you can easily create a good static model. But there are are many cases where a dynamic type works a lot better.

# August 19, 2010 3:46 AM

jeroenh said:

I really believe that dynamic does have it's place in C#. In many cases it's better to rely on static typing, but sometimes static typing just get's in your way (reflection, DOM-style API, XML, ...).

I like how you can 'opt in' to dynamic in C#, simply by using the 'dynamic' keyword. It gives us best of both worlds. At the same time, the built-in dynamic features are pretty lame, so Clay fills in a gap here. This stuff *should* have been in the framework in the first place. It is what `dynamic`/ExpandoObject/whatever should have been.

Very nice work.

# August 19, 2010 4:42 AM

Tony Steele said:

I'm currently using ExpandoObject with JSON.net too serialize/de-serialize between json and dynamic objects, it makes handling interaction between a javascript client much easy.

But you library looks an even better match than ExpandoObject for handling JSON on the server.

# August 19, 2010 5:45 AM

Matt said:

Hear, hear. Great post.

# August 19, 2010 9:08 AM

Robin Debreuil said:

cool, thanks for the description - checking out Orchard now...

I'm certainly not advocating using attr["foo"] vs attr.foo, just I never really need either in the domain I work in. I'm sure it makes sense in domains like Orchard.

And you aren't saying people should use dynamic where they know things at compile time, so seems like we agree all around : ).

btw, cool name, it evoke 'written in stone' vs 'written in clay' with just four letters : )

# August 19, 2010 8:08 PM

ms440 said:

Way cool! Thanks for sharing this. I'll try Clay over the weekend - it looks like the domain requirements of the app I'm working on are quite similar to Orchard's...

# August 20, 2010 1:53 AM

ms440 said:

Is there any way I can find what props were already set on the Clay object?

In DynamicObject I could cast it to IDictionary<string, object> and get the Keys. Is something similar is possible in Clay?

Thanks!

# August 20, 2010 7:57 PM

Bertrand Le Roy said:

@ms440: yes, reflection capabilities need to be added. This being said, I think you should be able to cast a Clay object into IDictionary already.

# August 20, 2010 8:01 PM

Mike said:

My head just exploded... is this C#?

# October 29, 2010 3:55 PM

Jace said:

Has anyone done some benchmark of Clay against Dictionary<string, object> and casting? Dynamic type is from my experience 3x slower than casting an object (if not more).

# April 15, 2011 4:06 AM

Bertrand Le Roy said:

@Mike: yes.

@Jace: not that I know of. Still, *what* dynamic types are you talking about? There are many ways that you can implement a dynamic object. Clay is not especially optimized for performance and uses dictionaries inside. Other implementations use expression trees, a method that is much faster than dictionaries and almost as fast a statically typed code.

In the end, using dynamic objects means sacrificing some performance for extreme flexibility. Ruby, Python and JavaScript developers usually consider it's a tradeoff they're willing to make. If to you performance is the most important maybe you should switch to assembly language. Kidding.

# April 15, 2011 2:26 PM

Richard Hein said:

Very nice and very relevant.  This is exactly the kind of thing C# needs for things that just can't be statically typed.  I absolutely love the fact that you can assign a ... Clay object? ... to an interface.  Then just carry on statically.

# May 6, 2011 11:17 PM

Michael said:

A big circle, isn't it? .NET slowly becomes a dynamic scripting language ... ;) This aside, what about dynamic get/set methods for a property and what about some access to the instance, I should probably read on the codeplex first, but I felt like posting a comment ;)

# January 22, 2012 1:42 PM

Fred said:

How do I find all the dynamic properties at runtime? to access them, it's fine with your example you know firstname & lastname, but what happens when even the property names are set at runtime?

TIA

Fred

# September 9, 2012 8:26 PM

Bertrand Le Roy said:

@Fred: it's up to each behavior to implement this or not. The Array behavior for example can be cast to IEnumerable.

# September 10, 2012 12:40 PM