F# and the Dynamic Lookup Operator ala C#
In the previous post, we covered various scenarios around how we’d make the syntax around using the MongoDB C# Driver a little nicer and less stringy. And before that we looked at using and abusing these so called dynamic lookup operators. In the F# language, we have the ability to define two “dynamic” operators, a get member operator denoted by the ( ? ), and the set member operator denoted by the ( ?<- ). The F# language and its associated libraries do not have an actual implementation of these operators, but instead allow you to implement them as you see fit. Previously, we tried two approaches…
Where We’ve Come From
We covered two basic scenarios, both of which have their plusses and minuses. The first approach was to wrap all calls to the get ( ? ) and set ( ?<- ) operators around the use of an IDictionary instance. The idea is that we treat the keys in the hash as if they were actual public members of our class. We can enable this behavior by implementing the operators as such:
let (?) (target : IDictionary<string,'Value>) targetKey = target.[targetKey] let (?<-) (target : IDictionary<string, 'Value>) targetKey value = target.[targetKey] <- value
That allows for some simple behavior such as getting and setting values in a Dictionary.
let d = Dictionary<string, Dictionary<string, string>>() d?Value1 <- Dictionary<string, string>() d?Value1?Value2 <- "Dynamic!" printfn "Value: %s" d?Value1?Value2 // prints Value: Dynamic!
This could be useful in quite a few scenarios, especially given that the MongoDB C# Driver’s Document class is based off an IDictionary. Just as well, you might imagine dealing with configuration values in a more DSL like fashion instead of indexer property syntax. But, I think we could do better than this.
The next idea was to use member restrictions to put some syntactic sugar around indexer properties. This way, any class which has the Item or this property in C#, could take advantage. As you may remember, member restrictions are a way of providing a form of duck typing, which is to allow us to restrict parameters based upon their method signatures. We could enable this kind of feature by implementing the operators as such:
let inline (?) this key = ( ^a : (member get_Item : ^b -> ^c) (this,key)) let inline (?<-) this key value = ( ^a : (member set_Item : ^b * ^c -> ^d) (this,key,value))
And once again, this enables our same example above as the IDictionary<Key,Value> has an Item property. But this comes with quite a few downsides that I’ve already mentioned, and in fact the F# compiler warms that you might be stepping into trouble while doing so. So, what other options do we have?
Where We’re Going
What about some of the features that are coming in .NET 4.0 revolving around the System.Dynamic namespace? Could we in fact take advantage of some of the work done there to enable dynamic behavior in our operators? The answer is unsurprisingly yes, although there is a little work, so prepare to get your hands a little dirty. In order to make this work, we’re going to need to understand how dynamic works in C# 4.0 as well as a bit about the Dynamic Language Runtime (DLR). So, read up there just a little bit and come back. I’ll wait… Just as well, we’re going to need two classes in particular, System.Runtime.CompilerServices.CallSite<T> and Microsoft.CSharp.RuntimeBinder.Binder, so best to study what they are what and they do (albeit limited in documentation as they are).
For example, if we have an object which inherits from System.Dynamic.DynamicObject and we’d like to use the TryGetMember and TrySetMember, how could we do it?
type DynamicDictionary() = inherit DynamicObject() let dictionary = new Dictionary<string, obj>() override __.TryGetMember(binder : GetMemberBinder, result : byref<obj>) = if dictionary.ContainsKey binder.Name then result <- dictionary.[binder.Name] true else base.TryGetMember(binder, &result) override __.TrySetMember(binder : SetMemberBinder, value : obj) = dictionary.[binder.Name] <- value true
Or just as well, could we enable the behavior on the System.Dynamic.ExpandoObject as well?
Let’s first tackle the get member operator. To enable this feature, we’re going to use the work that the C# team has done for us. What I mean by that is that we’re going to use the binders that C# uses for the DLR. So, if you’re following along at home with me, add a reference to Microsoft.CSharp.dll or observe in the code below and open the following namespaces.
#if INTERACTIVE #r "Microsoft.CSharp.dll" #endif open System open System.Dynamic open System.Linq.Expressions open System.Reflection open System.Runtime.CompilerServices open Microsoft.CSharp.RuntimeBinder open Microsoft.FSharp.Reflection
The next thing we’re going to do is create our method signature which takes in our object to invoke and a string target member, and we’re going to return a ‘TargetResult. This result could be simply a property name such as “Value”, or it could be an actual method such as Substring(0, 3).
let (?) (targetObject : obj) (targetMember:string) : 'TargetResult = let targetResultType = typeof<'TargetResult>
Once we have established the type, we can take one of two paths. Either our result type is a value such as string, integer, etc, or it is a function, and the two of them must be handled differently. Let’s take the case of the value return first.
if not (FSharpType.IsFunction targetResultType) then let argInfos = [| CSharpArgumentInfo.Create(CSharpArgumentInfoFlags.None, null) |] let getMemberBinder = Binder.GetMember(CSharpBinderFlags.None, targetMember, null, argInfos) let cs = CallSite<Func<CallSite, obj, obj>>.Create(getMemberBinder) cs.Target.Invoke(cs, targetObject) |> unbox
To determine whether our result type is not a function, we can call the FSharpType.IsFunction method from the Microsoft.FSharp.Reflection namespace. Inside, we create an array of CSharpArgumentInfos with no additional information nor name. Next, we create a get member binder with no additional information, our target member, no context, and our argInfos from above. After that we create our CallSite of a function that takes a CallSite and an object and returns an object with our getMemberBinder. Finally, we invoke the target of our CallSite with our CallSite and our target object and cast it to our ‘TargetResult. With this functionality implemented so far, we’ve enabled the following:
let len : int = "dynamic"?Length let now : DateTime = DateTime?Now let e = System.Dynamic.ExpandoObject() e?Dynamic <- "Totally"
But not all of our dynamic invocations are going to be simple properties and since we need to enable actual functions, we need to follow a slightly more difficult path. Because F# treats functions differently than C# does, we have to create a translation layer between the languages. Therefore a little bit of work is ahead of us.
Let’s start down the else path. First we need to get the domain type from our function elements. This domain type could be either a tuple of multiple arguments, null (or an empty tuple) or just a single argument. We then create a domain types array based upon our domain type to ensure that we have all the parameter types.
else let (domainType, _) = FSharpType.GetFunctionElements targetResultType let domainTypes = if FSharpType.IsTuple domainType then FSharpType.GetTupleElements domainType elif domainType = typeof<unit> then [| |] else [|domainType|]
Next, we need to create a function which actually handles the handles the dynamic invocation of our target member. This is rather long code so we’ll step through it carefully. I have comments along the way to help you navigate your way. First we need to get the arguments to our function from our domain types from above. If our domain types indicate that there are no values, then we yield nothing, else if one type, wrap our argument in the array, else if it’s a tuple, then break the tuple apart into an array. Secondly, like in our value scenario, we need to create our CallSite, and call Invoke on the Target field (note field and not property) with our binder, but we can only do this via reflection because we’re dealing with a CallSite and not a CallSite<T> since the Target is not exposed via the CallSite, but only its generic counterpart. Just as well, we’re doing a lot of this via reflection because we don’t know what our types will be until runtime anyhow.
let objToObjFunction = (fun argObj -> // Get the real argument values let realArgs = match domainTypes with | [| |] -> [| |] | [| argTy |] -> [| argObj |] | argTys -> FSharpValue.GetTupleFields(argObj) // Get our function type for the CallSite let funcTypeArgs = [| yield typeof<CallSite>; yield typeof<obj>; yield! domainTypes; yield typeof<obj> |] let funcType = Expression.GetFuncType funcTypeArgs let cty = typedefof<CallSite<_>>.MakeGenericType [| funcType |] // Create our CallSite let createFlags = BindingFlags.Public ||| BindingFlags.Static ||| BindingFlags.InvokeMethod let targetMemberArgs = Array.create (realArgs.Length + 1) (CSharpArgumentInfo.Create(CSharpArgumentInfoFlags.None, targetMember)) let createArgs = [|(box(Binder.InvokeMember(CSharpBinderFlags.None, targetMember, null, null, targetMemberArgs)))|] let cs = cty.InvokeMember("Create", createFlags, null, null, createArgs) |> unbox<CallSite> // Get the target for our CallSite let target = cs.GetType().GetField("Target").GetValue(cs) // Invoke our CallSite target let invokeFlags = BindingFlags.Public ||| BindingFlags.Instance ||| BindingFlags.InvokeMethod let invokeArgs = [| yield cs; yield targetObject; yield! realArgs |] target.GetType().InvokeMember("Invoke", invokeFlags, null, target, invokeArgs))
Finally, we need to create an F# function which invokes our object to object function from above with our desired return type and return it to the caller.
FSharpValue.MakeFunction(targetResultType,objToObjFunction) |> unbox<'TargetResult>
Once this is complete, we now have the capability of invoking both properties and methods just as if they were normal. For example, we can get a substring of a word and also check its length.
let sub : string = "dynamic"?Substring(0,3) let len : int = "dynamic"?Length
Now let’s address the dynamic set operator. Luckily this is a much simpler operator to implement as we needn’t worry about the other case of a function, and instead can concentrate solely on setting a property. First, let’s look at the signature required for this operator:
let (?<-) (targetObject : obj) (targetMember : string) (args : 'Args) : unit =
As you can see, this operator takes a target object, a target member and some arguments of type ‘Args and returns unit (void). Now, let’s look at the actual implementation. First, we need to create a set member binder with our binder flag with no additional information, our target member name, our target object type and our list of argument information. Next, we create a setter CallSite of type Func<obj, ‘Args, obj> which is to say that it takes our targetObject and our arguments and returns an object. Finally, we invoke the setter CallSite’s target with our setter CallSite, our target object and our arguments and ignore the return value.
let flags = CSharpArgumentInfoFlags.Constant ||| CSharpArgumentInfoFlags.UseCompileTimeType let argumentInfos = [| CSharpArgumentInfo.Create(CSharpArgumentInfoFlags.None, null); CSharpArgumentInfo.Create(flags, null) |] let binder = Binder.SetMember(CSharpBinderFlags.None, targetMember, targetObject.GetType(), argumentInfos) let setterSite = CallSite<Func<CallSite, obj, 'Args, obj>>.Create(binder) setterSite.Target.Invoke(setterSite, targetObject, args) |> ignore
For a complete source listing of both the get and set dynamic lookup operators, please refer to this Gist on my GitHub.
Having this in place, we’re able to both get and set values on such things as an ExpandoObject, our DynamicDictionary from above, and even custom classes such as an F# record.
// Expando object example let e = ExpandoObject() e?Property1 <- ExpandoObject() e?Property1?Property2 <- "Expando man!" printfn "%s" e?Property1?Property2 // Dynamic dictionary let d = DynamicDictionary() d?Dynamic <- true printfn "Dynamic? %b" d?Dynamic // Record type type Person = { Name : string; mutable Age : int } let me = { Name = "Matt"; Age = 22 } me?Age <- me?Age + 1 printfn "What's my age again? %d" me?Age
With this in place, the possibilities are expanded just a little bit. You might imagine a lot of the upcoming code such as the MongoDB C# Driver moving towards dynamic to provide some syntactic sugar, or even such things as Divan for CouchDB as well.
Do we also get the goodies of dealing with COM as well? The answer there is of course we do. For example, I’ll show the difference between the old and new ways of dealing with Excel. Instead of having to cast our worksheet to the _Worksheet interface, we get instead use the ? operator to say we know it’s there and I don’t feel like casting to it.
#if INTERACTIVE #r "Microsoft.Office.Interop.Excel.dll" #endif open Microsoft.Office.Interop.Excel let app = ApplicationClass(Visible=true) // Old let sheet1 = app.Workbooks.Add().Worksheets.[1] :?> _Worksheet sheet.Range("A1").Value2 <- "12345" // New let sheet2 = app.Workbooks.Add().Worksheets.[2] sheet2?Range("A1")?Value2 <- "12345" app.Quit()
So, as you can see, we get most of the benefits of what C# has, although they do a bit more optimization than I have done here such as caching.
Conclusion
Implementing these dynamic operators for using the C# dynamic binders was not trivial and took a little understanding of how the DLR works and what magic the C# compiler does to make this happen. These are not well documented APIs from the C# binder perspective, so understanding a bit of this can be tricky.
One question that could come from this post would be, should this be the official implementation for F#, just as there is some compiler magic for C#? And would it make sense to have a separate operator for doing more “custom” dynamic operations such as our IDictionary lookups among other items? Through the use of Modules, certainly you could decide which operator implementation you which to use, which would then allow us to have many implementations of our operator and we decide which one we need.