Announcing FsTest - A Testing DSL for F#

Over the past couple of months, I've been working on some language oriented programming concepts in F# to prove how Domain Specific Languages could work.  This coincided with my lofty goals of working towards more Behavior Driven Development (BDD) frameworks and expressiveness that I think is needed to make them work.  So, this led me down a road of looking first at assertion syntax.  With that, it made me create FsTest which is now available on CodePlex.  This is meant to be a beginning of a conversation on doing better assertions through functional programming and language oriented programming and by no means an end to it.

The Starting Point

During some of my research, I came across a now defunct project on Google Code called FsUnit.  The idea behind this project was to put a syntactic wrapper around NUnit to make assertions a bit more readable and to use unique features to F# to make this happen.  To be able to write such things as this for my assertions seemed powerful:

1 |> should (equal 1)
true |> should (be True)

And so on...  Ultimately, I thought it could be improved and expanded, so I talked to the project owner, Ray Vernagus, and took some of the ideas and ran with them.

Looking at FsTest

I had a couple of issues with the code as it was.  But, overall the ideas were quite sound.  I also wanted to use xUnit.net as the back end testing framework as I'm pretty comfortable with it.  The support for tests on static classes, and the ability to assert on throwing exceptions were some of the pluses that made me move in this direction.  Listed below are some of the assertions that are supported:

Testing equality:
"foo" |> should equal "foo"
"foo" |> should not_equal "bar"
null |> should be Null
"foo" |> should be NonNull
"foobar" |> should contain "foo"
"foobar" |> should contain "hello"

Testing True/False:
true |> should be True
false |> should be False

Testing collections:
[1..10] |> should be NonEmpty
[] |> should be Empty
[1..10] |> should have 3
[1..10] |> should not_have 25

Exception Management:
throwsExceptionFunc |> should (throw_exception<ArgumentException>)
doesntThrowException |> should not_throw_exception

And some full examples might look like this:

#light

open System
open Xunit
open FsxUnit.Syntax

let throwsException() : unit =
  raise(ArgumentException "Bad things!")

[<Fact>]
let throwsException_should_throw_exception() =
  throwsException |> should (throw_exception<ArgumentException>)

[<Fact>]
let value_in_range_should_be_in_range() =
  2 |> should be_in_range 0 100

It's a work in progress, but I feel that I have most of the assertion cases covered in this DSL.  The ability through partially applied functions really shone when creating this library. 

The Concepts Behind It

What makes this work syntactically is the use of the forward operator.  I've covered this briefly in previous posts, but I'll elaborate on this again.  This is one of the most important operators available to us in the F# language.  This is the ability to invert the arguments in such a manner as the first argument is declared first and then the function statement is last.

This is how it's declared in F#:

let (|>) x f = f x

Which allows me to better express my intent and gives me the two advantages:
  • Clarity of intent - allows you to perform various operations on the data in a forward chaining way such as:
    let length = [1..10] |> List.map(fun x -> x * x * x) |> List.filter(fun x -> x % 3= 0) |> List.length

  • Type inference - allows type information to flow from the left to the right and allows me to express my data first as part of my method.  Shows better intention this way.

And similarly, something in C# would use extension methods to add a forward method, or maybe in this case a Should method which will then take a function as the second parameter, and then overload it as need be.

Using Actions might look something like this:

public static void Should<TArg1, TArg2>(this TArg1 arg1, Action<TArg1, TArg2> action, TArg2 arg2)
{
    action(arg1, arg2);
}

And conversely to use the Func delegate might look like this:

public static TResult Should<TArg1, TArg2, TResult>(this TArg1 arg1, Func<TArg1, TArg2, TResult> func, TArg2 arg2)
{
    return func(arg1, arg2);
}

public static void Contain<T>(IEnumerable<T> items, T item)
{
    Assert.Contains(item, items);
}

And to use it would look something like this:

Enumerable.Range(1, 10).Should(Contain, 3);

But, I'm not necessarily a fan of that style of syntax.  C# isn't quite a functional language which allows me to do such clever things with operators.  I don't think it will ever gain that level of expresiveness required.

Instead, I created two functions that could be chained together such as the should function and the be function.  This allows me to create some of the examples above.  Let's take a look at each function:

let should f actual =
  f actual

let be (f : 'a -> unit) actual =
  f actual

So, that will allow me to express such things as:

true |> should be True

And the true function might look something like this:

let True condition =
  Assert.True(condition)

As you can see, there isn't much to this, and covers most of the assertion scenarios that I've encountered.

Where To Go From Here?

So, where am I headed with this?  For right now, I'm happy with the assertion syntax I've been able to achieve with the language.  For the future, I'm looking at creating a more expressive way for doing BDD and the expresiveness of F# I think can handle such things.  I've been following the Specs project for a little bit and I think they have a few good things going for them, although Scala isn't a functional language in the way that F# is, so I'm not sure how many lessons can be applied.  Specter is another interesting one as well as Machine Specifications (MSpec) from the one and only Aaron Jensen.

I'd like to thank the F# team, Dustin Campbell and Harry Pierson (DevHawk) for their help on this as well.  The team around F# is a wonderful team and I can't wait for the CTP of F# to ship.  Makes me excited sometimes when working for Microsoft like I do.

But, in the mean time, feedback is appreciated.  Tell me what works and what doesn't.

kick it on DotNetKicks.com
Published Thursday, June 19, 2008 7:16 PM by podwysocki

Comments

# re: Announcing FsTest - A Testing DSL for F#

Wednesday, April 22, 2009 11:07 AM by Justin Bozonier

Hi Matthew!

I'm looking to start using F# for my BDD on C#. Have you been able to flesh this DSL out more?

# re: Announcing FsTest - A Testing DSL for F#

Sunday, April 26, 2009 4:01 PM by podwysocki

@Justin

No, I haven't touched it in a while, although you may want to check out FsStory and others in this space.  I'm more focused on things such as QuickCheck and Pex at the moment.

Matt

# re: Announcing FsTest - A Testing DSL for F#

Thursday, January 24, 2013 9:39 PM by Sabo

If you desire to take much from this article then you have

to apply such strategies to your won web site.