Contents tagged with Domain Specific Languages
-
Introducing Maestro – A DSL for Actor Based Concurrency
As you may have noticed from my blog, there is a big interest in concurrency. Along with that, I’ve made my way into Actor model based concurrency including such forays into Erlang, the Haskell-Actor package, Mailbox Processing in F#, and even such things as Retlang in C#. The problem with some of the .NET solutions is that they don’t go far enough into memory isolation and channel contracts as to what can and cannot happen. In order to enforce such a behavior, we need some sort of static analysis to indicate that. Of course with static analysis, it could be ignored and might not be as type rich as, say a DSL. And that’s where Maestro comes in.
-
Announcing FsTest - A Testing DSL for F#
Over the past couple of months, I've been working on some language oriented programming concepts in F# to prove how Domain Specific Languages could work. This coincided with my lofty goals of working towards more Behavior Driven Development (BDD) frameworks and expressiveness that I think is needed to make them work. So, this led me down a road of looking first at assertion syntax. With that, it made me create FsTest which is now available on CodePlex. This is meant to be a beginning of a conversation on doing better assertions through functional programming and language oriented programming and by no means an end to it.
The Starting Point
During some of my research, I came across a now defunct project on Google Code called FsUnit. The idea behind this project was to put a syntactic wrapper around NUnit to make assertions a bit more readable and to use unique features to F# to make this happen. To be able to write such things as this for my assertions seemed powerful:
1 |> should (equal 1)
true |> should (be True)
And so on... Ultimately, I thought it could be improved and expanded, so I talked to the project owner, Ray Vernagus, and took some of the ideas and ran with them.
Looking at FsTest
I had a couple of issues with the code as it was. But, overall the ideas were quite sound. I also wanted to use xUnit.net as the back end testing framework as I'm pretty comfortable with it. The support for tests on static classes, and the ability to assert on throwing exceptions were some of the pluses that made me move in this direction. Listed below are some of the assertions that are supported:
Testing equality:
"foo" |> should equal "foo"
"foo" |> should not_equal "bar"
null |> should be Null
"foo" |> should be NonNull
"foobar" |> should contain "foo"
"foobar" |> should contain "hello"
Testing True/False:
true |> should be True
false |> should be False
Testing collections:
[1..10] |> should be NonEmpty
[] |> should be Empty
[1..10] |> should have 3
[1..10] |> should not_have 25
Exception Management:
throwsExceptionFunc |> should (throw_exception<ArgumentException>)
doesntThrowException |> should not_throw_exception
And some full examples might look like this:
#light
open System
open Xunit
open FsxUnit.Syntax
let throwsException() : unit =
raise(ArgumentException "Bad things!")
[<Fact>]
let throwsException_should_throw_exception() =
throwsException |> should (throw_exception<ArgumentException>)
[<Fact>]
let value_in_range_should_be_in_range() =
2 |> should be_in_range 0 100
It's a work in progress, but I feel that I have most of the assertion cases covered in this DSL. The ability through partially applied functions really shone when creating this library.
The Concepts Behind It
What makes this work syntactically is the use of the forward operator. I've covered this briefly in previous posts, but I'll elaborate on this again. This is one of the most important operators available to us in the F# language. This is the ability to invert the arguments in such a manner as the first argument is declared first and then the function statement is last.
This is how it's declared in F#:
let (|>) x f = f x
Which allows me to better express my intent and gives me the two advantages:
- Clarity of intent - allows you to perform various operations on the data in a forward chaining way such as:
let length = [1..10] |> List.map(fun x -> x * x * x) |> List.filter(fun x -> x % 3= 0) |> List.length
- Type inference - allows type information to flow from the left to the right and allows me to express my data first as part of my method. Shows better intention this way.
And similarly, something in C# would use extension methods to add a forward method, or maybe in this case a Should method which will then take a function as the second parameter, and then overload it as need be.
Using Actions might look something like this:
public static void Should<TArg1, TArg2>(this TArg1 arg1, Action<TArg1, TArg2> action, TArg2 arg2)
{
action(arg1, arg2);
}
And conversely to use the Func delegate might look like this:
public static TResult Should<TArg1, TArg2, TResult>(this TArg1 arg1, Func<TArg1, TArg2, TResult> func, TArg2 arg2)
{
return func(arg1, arg2);
}
public static void Contain<T>(IEnumerable<T> items, T item)
{
Assert.Contains(item, items);
}
And to use it would look something like this:
Enumerable.Range(1, 10).Should(Contain, 3);
But, I'm not necessarily a fan of that style of syntax. C# isn't quite a functional language which allows me to do such clever things with operators. I don't think it will ever gain that level of expresiveness required.
Instead, I created two functions that could be chained together such as the should function and the be function. This allows me to create some of the examples above. Let's take a look at each function:
let should f actual =
f actual
let be (f : 'a -> unit) actual =
f actual
So, that will allow me to express such things as:
true |> should be True
And the true function might look something like this:
let True condition =
Assert.True(condition)
As you can see, there isn't much to this, and covers most of the assertion scenarios that I've encountered.
Where To Go From Here?
So, where am I headed with this? For right now, I'm happy with the assertion syntax I've been able to achieve with the language. For the future, I'm looking at creating a more expressive way for doing BDD and the expresiveness of F# I think can handle such things. I've been following the Specs project for a little bit and I think they have a few good things going for them, although Scala isn't a functional language in the way that F# is, so I'm not sure how many lessons can be applied. Specter is another interesting one as well as Machine Specifications (MSpec) from the one and only Aaron Jensen.
I'd like to thank the F# team, Dustin Campbell and Harry Pierson (DevHawk) for their help on this as well. The team around F# is a wonderful team and I can't wait for the CTP of F# to ship. Makes me excited sometimes when working for Microsoft like I do.
But, in the mean time, feedback is appreciated. Tell me what works and what doesn't.
- Clarity of intent - allows you to perform various operations on the data in a forward chaining way such as:
-
DC ALT.NET May Wrapup - Common Lisp and Applying Lessons Learned
Last night's DC ALT.NET meeting was a great success. We had Craig Andera, of PluralSight and FlexWiki fame, talk to us about Common Lisp and some of the lessons he learned. It was great to see the guys from the FringeDC group join us as well. I can definitely see a lot of overlap between the two groups as we both struggle to find new and innovative ideas for solving our hardest problems. We tend to look outside of our community to find what has worked and what hasn't worked for each community. Because at the end of the day, we're all developers with just different backgrounds. I want to thank the Motley Fool for hosting the event for us, and we'd love to come back if you'll have us.
The Presentation
Craig spent much of his summer vacation actually learning Common Lisp. The original idea was to learn Ruby, but why not go back to the grandfather of them all, Lisp. I remembered Lisp from back in the college years, but had forgotten most of it. It's very interesting to watch the presentation and learn how flexible of a language it is since we're just dealing with expression trees. I can definitely see where other languages got their heritage. All good things tend to come back to Lisp and SmallTalk!
Anyhow, the important features we covered were object oriented programming with Common Lisp Object System (CLOS), macros, defining properties and methods, and even the .NET interop story with IronScheme and others. It was a great time and I learned a lot. If you wish to grab his presentation notes, you can find them here.
Next time, I'll be presenting F#, so we'll keep on the functional programming style with some OOP mixed in, so I hope we see another great crowd for that. As always, join the mailing list here if you want to learn more.
-
Looking at DSLs in .NET
As I've mentioned in recent posts such as here, here and here, I've been very interested in Domain Specific Languages (DSLs), especially with regards to F# and the DLR as well. I recently re-listened to Software Engineering Radio Episode 52 with Obie Fernandez discussing DSLs in Ruby. One of the things that attracted me to Ruby for this was the flexibility of the syntax for closures, mixins, etc. Anyhow, it's a good listen and if you're new to the subject, you should give it a go. Also, there is a slide deck of DSLs in Ruby which accompanies this episode which can be found here.
So, of course this gets me excited about the possibilities of seeing such things in IronRuby. After seeing John Lam's presentations at MIX08 and listening to him on various podcasts, I'm excited that they are making such progress and hopefully get it into our hands soon.
But, before we get too deep into things, I just want to take a step back and look quickly at what DSLs are.
Internal and External DSLs?
So, what are DSLs? Well, to put it succinctly, it's a small language that's used for a very narrow task. You can think of these as languages specific to a domain such as medical claims processing, stock trading and so on that only have meaning there. These languages mean very little outside their problem domain and probably wouldn't make sense to anyone outside. I'm very well aware of such things as I've worked in the medical claims processing industry and their terms, calculations and so on are very specific and to solve the problem well, it's best to suit the language best for expressing solutions.
Martin Fowler wrote an article entitled "Language Workbenches: The Killer-App For Domain Specific Languages?" in which he talks about the history of DSLs especially in the Lisp world, but until now really haven't caught on. Martin argues that XML structures such as configuration files, and so on qualify for that status, due to the fact that it is readable by a human and probably a domain expert as well. But Lisp is well know for the Lex/Yacc parsing and expression trees and so on.
Now the real interesting part comes in when we talk about internal versus external DSLs...
External DSLs, quite simply, are those languages that are not in the same language as the main application itself. This means that I could be free to write any free-form code I wish in order to suit my domain specific need. This means that you need to write parsers and then ultimately would need to have a translation boundary between your DSL and your application. This is where I think something like the Dynamic Language Runtime (DLR) could come into play. What I mean by that is that if you write your language parser for the DLR, and I'll get into that shortly. There is a bit of overhead with this of course, plus a good debugger and IDE, but with time and patience things like this can be overcome. Extending such things as #Develop to encompass those pieces is feasible.
Internal DSLs, on the other hand, are the little languages you can create inside your current language of choice. Now languages such as Lisp, Ruby, Scala, Boo, and F# seem a bit more suited for these than the mainstream languages of C++, C# and Java. Of course one of the bigger obstacles is the pesky curly brace which Ruby allows you to discard. F# doesn't have a concept of this either, and instead the indentation scopes the values and functions and so on. Martin has an interesting DSL written in Java that's interesting and could be better applied in different languages.
Thinking About The DLR
As I said before, I'm pretty excited about the DLR and the flexibility it can give me as a software engineer looking for new and better ways to solve my customer's problems. Not only that, but I'm a language geek at heart, what can I say? I've posted several items on building on the DLR in the context of external DSLs as well as writing custom compilers for .NET. Projects such as Irony also appeal to me in that way.
If you want to play around with the DLR, you can get it in one of two places, the IronPython download on CodePlex or on RubyForge with the IronRuby project.
Martin Maly, a member of the DLR team has continued his posts about Building on the DLR. He took some time off to work on some DLR related issues and now is back with some more posts. Let's continue where we left off last time:
- Answering Some Questions
Martin talks about Static and Dynamic Nodes on DLR Trees
- Variations on Trees
Martin talks about DLR trees and LINQ Expression trees
- More questions
Martin talks about implementing a Print method that acts differently per incoming data type
- Extension Methods
Martin talks about adding extension methods to the ToyScript sample
DSLs In Boo?
Oren Eini, aka Ayende, has been working on a book about DSLs called "Building Domain Specific Languages in Boo". Recently, he posted about some sample chapters now available online with the source code. I highly recommend that you at least give it a look. You now have access to the Early Access Edition, and of course you can buy it online. It'll be interesting to see Ayende at ALT.NET Open Spaces, Seattle to see if he wants to cover more of this stuff.
If you're not familiar with Boo, it's one of Ayende's preferred languages. Such tools as Binsor (The DSL for Windsor) were written in Boo. If you're not familiar with the language, it has very similar syntax to Python and formats very nicely. This easily fits Martin Fowler's category as an external DSL. #Develop has some support for Boo to make it a first class language in the .NET family. It also ships as part of the Castle Project. Anyways, a quick sample of Binsor makes it look like a nice DSL for type registration in Castle Windsor in Boo Script:
import Rhino.Commons
logger = Component("console_logger", ILogger, ConsoleLogger)
Check out the first chapter of Ayende's book which is available free online and explore it yourself for DSLs.
DSLs in F#?
Another part that had me intrigued was the possibilities of not only Boo to do this, but F# as well. Robert Pickering covers this in his book, Foundations of F#. If you pay attention to Chapter 11, he covers DSLs in F# and gives a few examples of how you can do so. To me, it's pretty powerful because you have a lot of the built-in features of a functional programming language such as lists, pattern matching and so on. Such examples as given are such things as the arguments parser in F# that is described in the Arg class in Microsoft.FSharp.Compatibility.OCaml namespace. This allows you to parse well known data structures as first class citizens. Martin Fowler also gave such an example from the article I quoted from above. F# lends itself quite well to DSLs in regard to support for lambda expressions.
Don Syme also covers these topics in his book Expert F# in Chapter 9. This covers more language oriented programming techniques, but you can scan some information about building DSLs in F#. Some interesting parts of this come down to Active Patterns which I covered partially yesterday. In the coming weeks, I hope to post some of my forays into this, taking some samples from the Ruby community and applying some of the same functionality in F#.
Conclusion
There is still much yet to be covered in this topic of DSLs in .NET languages. We can go on and on with regards to internal and external DSLs and argue about which language is suited for each. But in the coming weeks, I hope to take some samples and show how they can apply cleanly in F#, and probably run into some language problems where it might not be the best fit. But, that's the fun part about it.
- Answering Some Questions
