August 2008 - Posts
I presented MbUnit at the Manchester ThoughtWorks geek night last Wednesday, it was a great night with lots of great questions and feedback. It was really great meeting folks who use MbUnit day to day, it really validates the work we do on the project. I think for DDD I'll change the format of my talk as I have now started thinking about a story format for the session, voting for that is due to close soon so we shall see. The next post in the MbUnit series will be about the combinatorial test. Let's take a look at the object under test first.
Starting with the interface
1: public interface IAlphaString
2: { 3: bool ValidString(string inputString);
4: }
and now the concrete class
1: using System.Text.RegularExpressions;
2:
3: public class AlphaString : IAlphaString
4: { 5: public bool ValidString(string inputString)
6: { 7: Regex r = new Regex("[a-zA-Z]", RegexOptions.None); 8: return r.IsMatch(inputString);
9: }
10: }
This class tests for alpha strings and returns false if the test fails, true if the test passes. Before we get into testing this class we need one more class
1: using System;
2:
3: public class StringHolder
4: { 5: public string StringValue;
6: public bool IsValid;
7:
8: public StringHolder(string stringvalue, bool isValid)
9: { 10: StringValue = stringvalue;
11: IsValid = isValid;
12: }
13:
14: public override string ToString()
15: { 16: return String.Format("{0},{1}", StringValue, IsValid); 17: }
18: }
This class allows to set a test value and expected result. Wiring this object to the object under test we do the following.
1: using System.Collections;
2: using MbUnit.Framework;
3:
4: [TestFixture]
5: public class IAlphaStringTest
6: { 7: [Factory(typeof(IAlphaString))]
8: public IEnumerable AlphaStringInstances()
9: { 10: yield return new AlphaString();
11: }
12:
13: [Factory(typeof(StringHolder))]
14: public IEnumerable TestStrings()
15: { 16: yield return new StringHolder("aaaa", true); 17: yield return new StringHolder("111", false); 18: }
19:
20: [CombinatorialTest]
21: public void AlphaValuesTest([UsingFactories("TestStrings")] StringHolder testValues, 22: [UsingFactories("AlphaStringInstances")] IAlphaString itemstoTest) 23: { 24: Assert.AreEqual(itemstoTest.ValidString(testValues.StringValue), testValues.IsValid);
25: }
26: }
First we deploy the MbUnit factory attribute to have a referance to object under test and test holder object(s), we then link the factories into the actual test. The end result is that for each test value\expected result pair we run the test value against the obect under test and compare the expected result to the actual result. Let's take a look at v3 version
1: using System.Collections.Generic;
2: using MbUnit.Framework;
3:
4: [TestFixture]
5: public class CombinatorialTest
6: { 7:
8: public IEnumerable<IAlphaString> GetInstances()
9: { 10: yield return new AlphaString();
11: }
12:
13: [Test]
14: [Row("aaaa", true)] 15: [Row("111", false)] 16: public void Test(string value, bool isValid, [Factory("GetInstances")] IAlphaString alphaString) 17: { 18: Assert.AreEqual(isValid, alphaString.ValidString(value));
19: }
20: }
V3 makes things simpler in that we no longer need a test holder object and instead we can pair the test value\expected result to the object under test within the row test. As I am hopefully showing we are taking the long standing row fixture to the level in v3.
Factories in MbUnit work in much the same way in v2 (where they have been available since the start) and v3, in v2 we would use a Factory as follows.
1: using System.Collections;
2: using MbUnit.Framework;
3:
4: public class ArrayListFactory
5: { 6: [Factory]
7: public ArrayList ProviderEmptyArrayList
8: { 9: get { return new ArrayList(); } 10: }
11:
12: [Factory]
13: public ArrayList ProviderArrayList
14: { 15: get
16: { 17: ArrayList list = new ArrayList();
18: list.Add(0);
19: list.Add(1);
20: return list;
21: }
22: }
23: }
As seen in the TypeFixture example this marks methods as Factory functions that can be reused across tests, for example by a ProviderFactory marked test etc. They are useful way of decorating methods as data providers rather than test methods. Let's look at a v3 example.
1: using System;
2: using System.Collections;
3: using MbUnit.Framework;
4:
5: [TestFixture]
6: public class FactoryTest
7: { 8: public ArrayList ProviderArrayList
9: { 10: get
11: { 12: var list = new ArrayList { 1, 2 }; 13: return list;
14: }
15: }
16:
17: [Test]
18: [Factory("ProviderArrayList")] 19: public void Test (int value)
20: { 21: Assert.GreaterThan(value, 0);
22: }
23: }
MbUnit v3 Factory attributes work differently here in that we mark a given method as a Factory by marking it's given name rather than decorating the actual method, on this example we mark the ProviderArrayList method as a Factory method but decorate our Test method instead.
Some software approaches run in cycles
Gather requirements (sometimes)
Write Code
Test code (sometimes)
Get the requirements wrong and the requirements change midway through writing the code and your writing more code or changing code and the time scale goes up (yes even if the deadline is unmoveable the coders still have to work more hours). Worse yet they change at the testing level and it means more code and yet more testing. Expensive and time consuming when you could use all that to deliver sooner and more to your customer.
Let’s say the cycle takes 6 weeks, 2 weeks in each stage. On week 3 the customer asks you where you are and you either make them wait 2 weeks more or give them an incomplete version. They then ask for this, that and other, with the need to gather further requirements, write more code\change code the cycle is looking longer and longer and the customer is still not seeing what they asked for (and keeps changing their mind). Sounds like scope-creep sure enough but if the customer is not getting what they asked for are you delivering and will they keep paying you? Requirements sign off can help but will the customer agree and be happy that what they signing off is what they asked for (and keep paying you for it). They may have agreed to Black but they ment #001.
Agile has the answer though....
No one has yet said that Agile will save the day and our industry will be bright and wonderful, some agree that Agile works for some things but not others aka THE ENTERPRISE.
I don't personally agree as Agile is not any one thing, it is just a another way of doing things and if you have the foundations right you can adapt it your team\business etc. I don't have all the answers, if you write widgets that control the flow of ear wax in a 3D human head sim can't say what you will need to do. Only YOU can answer that question, you do the job, you know what works and want doesn’t but to cast Agile aside for less than serious reasons will keep you on the same late\failed\expensive project path.
Agile was not dreamed up by a collection of goofy nerds but was a collection of ideas from a group of people who had spent decades in the industry and created the concepts from those same decades spent on software projects (and seeing why projects failed).
Agile does fail too though, lots of stories about failed Agile projects with reasons ranging from incomplete\unworkable adapted Agile process through to failing to understand\follow Agile concepts. If you’re serious about it, hire an Agile coach, sure some cost the GNP of a small nation but they are worth their weight in gold in using their experience and skills to help you adapt and follow Agile processes.
Have you adopted agile, what worked\failed for you?
Martin Woodward has a great article on software estimation, why it is hard and what Agile brings to the table. The approach I have seen the most in the past is the grand old FITA analysis (Finger in The Air) estimate. Get it right and your on time, get it wrong all hell breaks loose (most of which cost money e.g the cost of late delivery, the cost of extra development etc). If the coders finish early then chances are Parkinsons law applies and still costs you. Projects even fail completely or the customer walks away never to be seen again, software estimation should be science and not an art, but that's what it is.
If you have not read the Agile Planning and Estimation book then you should, much of what I am going to talk about here is distilled from Mike Cohns book and is an invaluable read.
Activity based planning (gantt chart) folds in several tasks into a single activity e.g
create new shopping basket
A block of time is then set for that activity, FITA 4 days. There are many things wrong with this, the customer won't see any progression in those steps e.g.
create visual layout for basket
create add functionality
create edit\delete functionality
create summary
If the customer sees progression they will be happy with what they are getting and most criticially by breaking down the task into features you can estimate better.
Let's rank these first by what the customer wants to see first from 1-5, the customer may rank them all at 5 but only 1 feature can be ranked high.
5 create visual layout for basket
4 mcreate add functionality
4 create edit\delete functionality
3 create summary
In the first pass we know what the customer wants to see right away "create visual layout for basket". Agile works in time blocks (called iterations) these provide a set block of time so that everything is not done all at once, the customer can see progress and can ask for changes but at the end of iteration, you set how long each one is but they are intended to be short, 1 week at best. For a given week therefore we know anything we do can be no longer than a week (if so it may need further breaking down).
For each feature we do a certain amount of FITA (or any other approach you like) but with it broken down we try to base it on work we have done before on previous projects or iterations. Again we score based on time,
7 days 1
6 days 2
5 days 3
4 days 4
3 days 5
2 days 6
1 days 7
which gives us
6 create visual layout for basket
7 mcreate add functionality
7 create edit\delete functionality
7 create summary
Which gives us 5 days in total, a day longer than our first FITA approach (you may be thinking why the task takes a day longer, if 4 days is inaccruate then either the developers will work 2 days in 1 day to meet that deadline or the 4 days you give the customer will turn into the actual 5 days it takes). If we match these up to what our customer wants to see we know what we will deliver to them in that iteration by adding up the scores.
11 create visual layout for basket
11 mcreate add functionality
11 create edit\delete functionality
10 create summary
In the 7 day block we can deliver to the customer with 2 days to go. If the iteration was shortend (assuming other feature planning allows) to 5 days then the we can still deliver. Let's assume that the customer wants to see something in 4 days (again assumming other features allow) then you could still deliver the features they rank as important but the lesser important features slip to the next iteration, if the customer wants that feature then either the iteration remains at 5 days or the feature moves to the next iteration (which could be shorter still).
Your still guessing time for functionality though?
Yes you are but by being broken down into smaller tasks you can far more easily base them on previous tasks that may be associated to the task in question. If your guess is wrong and the task takes longer or is completed faster then you can adjust the task for next time. Looping in members of the team also makes the difference here, Agile involves the whole team in the planning experience and you should draw on their experince for how long features may take, planning poker can also help here.
How do you estimate software timescales, what works for you?
"You said no RowTest post!"
I know I did but this v3 feature I'd like to talk about, Row value headers, let's look at this example.
1: [Test]
2: [Row(1, 3, "test")]
3: [Row(2, 2, "test")]
4: public void Example(int sum1, int sum2, string value)
5: { 6: Assert.AreEqual(4, sum1 + sum2);
7: }
Here we have a row test with sample values (note you can mix types in this way) which seeds the test, so far so not unusual, let's try this however.
1: [Test]
2: [Row(1, 3, "test")]
3: [Row(2, 2, "test")]
4: public void Example(string value, int sum1, int sum2)
5: { 6: Assert.AreEqual(4, sum1 + sum2);
7: }
This test would fail as the Row seeds the test parameters in the order they are set in the Row, so now the string values are seeded with int values. While we could just set the parameters in the right order we can also do the following.
1: [Test]
2: [Header("sum1", "sum2", "value")] 3: [Row(1, 3, "test")]
4: [Row(2, 2, "test")]
5: public void Example(string value, int sum1, int sum2)
6: { 7: Assert.AreEqual(4, sum1 + sum2);
8: }
Same test parameter order as before but the test passes. The reason is the Header attribute, this states to MbUnit how each row test value relates to the test parameters so rather than change our test method we can instead fine tune the test data instead.
This is the first of series of posts I'll be running on MbUnit, I'll focus on v2 first and then show an example in v3. I won't focus on RowTest as this is a popular feature and is well known both by the MbUnit audience and by other frameworks. For the opening post I'll focus on the TypeFixture, let's take a look at the code.
1: using System;
2: using System.Collections;
3: using MbUnit.Framework;
4:
5: [TypeFixture(typeof(IEnumerable))]
6: public class TypeFixtureExample
7: { 8: [Provider(typeof(IEnumerable))]
9: public ArrayList ProviderEmptyArrayList()
10: { 11: return new ArrayList();
12: }
13:
14: [Provider(typeof(IEnumerable))]
15: public ArrayList ProviderArrayList()
16: { 17: ArrayList list = new ArrayList();
18: list.Add(0);
19: list.Add(1);
20: return list;
21: }
22:
23: [Test]
24: [ExpectedException(typeof(InvalidOperationException))]
25: public void CurrentCalledBeforeMoveNext(IEnumerable en)
26: { 27: IEnumerator er = en.GetEnumerator();
28: object p = er.Current;
29: }
30:
31: [Test]
32: [ExpectedException(typeof(InvalidOperationException))]
33: public void CurrentCalledAfterFinishedMoveNext(IEnumerable en)
34: { 35: IEnumerator er = en.GetEnumerator();
36: while (er.MoveNext());
37: object p = er.Current;
38: }
39: }
We have two tests that will run on both methods that are decorated by the Provider attribute, each provider set's up an ArrayList as our target type and each test runs against both providers. Each provider seeds the test method, you could also define one provider as your target type and any number of tests against that provider. If you want to break up your tests into reusable parts then you could do the following.
1: using System.Collections;
2: using MbUnit.Framework;
3:
4: public class ArrayListFactory
5: { 6: [Factory]
7: public ArrayList ProviderEmptyArrayList
8: { 9: get { return new ArrayList(); } 10: }
11:
12: [Factory]
13: public ArrayList ProviderArrayList
14: { 15: get
16: { 17: ArrayList list = new ArrayList();
18: list.Add(0);
19: list.Add(1);
20: return list;
21: }
22: }
23: }
and use these as follows
1: using System;
2: using System.Collections;
3: using MbUnit.Framework;
4:
5: [TypeFixture(typeof(IEnumerable))]
6: [ProviderFactory(typeof(ArrayListFactory), typeof(IEnumerable))]
7: public class ProviderFactoryTest
8: { 9: [Test]
10: [ExpectedException(typeof(InvalidOperationException))]
11: public void CurrentCalledBeforeMoveNext(IEnumerable en)
12: { 13: IEnumerator er = en.GetEnumerator();
14: object p = er.Current;
15: }
16:
17: [Test]
18: [ExpectedException(typeof(InvalidOperationException))]
19: public void CurrentCalledAfterFinishedMoveNext(IEnumerable en)
20: { 21: IEnumerator er = en.GetEnumerator();
22: while (er.MoveNext()) ;
23: object p = er.Current;
24: }
25: }
Here we break out the providers into Factories and referance them in our test using the ProviderFactory attribute. So unlike our first test the Providers are isolated from the test and reusable across other tests however we can still seed our tests as before.
Let's take a look at how MbUnit v3 does this.
1: using System.Collections.Generic;
2: using MbUnit.Framework;
3:
4: public class TypeFixtureExample
5: { 6: public static IEnumerable<IEnumerable> GetInstances()
7: { 8: yield return new ArrayList();
9: yield return new ArrayList { 0, 1 }; 10: }
11:
12: [Factory("GetInstances")] 13: public IEnumerable Instance;
14:
15: [Test]
16: [ExpectedException(typeof(InvalidOperationException))]
17: public void CurrentCalledBeforeMoveNext()
18: { 19: var er = Instance.GetEnumerator();
20: var p = er.Current;
21: }
22:
23: [Test]
24: [ExpectedException(typeof(InvalidOperationException))]
25: public void CurrentCalledAfterFinishedMoveNext()
26: { 27: var er = Instance.GetEnumerator();
28: while (er.MoveNext()) ;
29: var p = er.Current;
30: }
31: }
V3 has done away with the Provider attribute, we use Factory attributes as before but no longer need the ProviderFactory attribute as v3 is a little cleverer at wiring up tthe data. Our type tester methods remain the same however we seed them using the Factory attribute, this like the v2 attribute will create and hold test data for us. In this example the Factory attribute creates a enumerable list of our type along with seeded data. The GetInstances method plays the role of the Provider methods in the v2 example, we are using some 3.5 sugar to shorten the syntax here but the principle of seeded types remains the same.
Just a quick reminder about this Wednesday (27th) and the Manchester UK ThoughtWorks geek night where I will be giving a session on MbUnit v2, I'll also be showing some MbUnit v3 and Gallio bits as well. If you have not already signed up then check out the registration site.
I am also hoping to take an adapted version of this session down to DDD UK in November.
Jeff has also announced that he will be giving a MbUnit v3\Gallio session at the Silcon Valley code camp (November 8th\9th) and at QCon (November 18-21st), if your in the SF Bay area then be sure to check out his sessions.
I started pondering about the DLR and running it on the SSCLI when I came across this article on the CoreCLR (shared source version of that would be fun) and the following.
A lot of the Silverlight PAL benefits from lessons learned when we developed the Shared Source Common Language Infrastructure (SSCLI), also known as Rotor. The SSCLI ran on a number of UNIX-style platforms as well as Windows. The base OS functionality varies widely on UNIX-style platforms. The SSCLI PAL had to work on both microkernels (such as the Mach kernel in Mac OS X) and monolithic kernels and had to cope with different OS services such as threading, exception handling, and networking stacks. Because Silverlight only targets Windows and Intel Mac machines we were able to write Mac-specific implementations for many of the functions in the PAL, which helps with the size and performance of the PAL.
The diagram in the article makes more sense but the CoreCLR Platform Adaption Layer (PAL) works in the same way as Rotors in that anything Win32 related goes direct to the OS while anything else goes via the PAL. Rotors PAL targeted different chip sets and Win32, FreeBSD Unix and Mac OSX. Free BSD and the Mac OSX are based on the same kernal (the Mach kernal) so chip sets aside porting the Mac PAL to the Unix PAL was relatively stright foward. The CoreCLR PAL targets the same chip set's and only requires a small sub-set of OS functionlity which keeps the complexity of the CoreCLR's PAL down.
The SSCLI aka Rotor tends to not draw all that much attention unless your a researcher or CLR wonk but it is one of (if not the best) source code referances to the very depths of .NET that you will ever find. For me personally getting the very most of the framemwork I develop on every day is about expanding my understanding of how that framework works, sure the compilers and there tricks come next but the root is the CLI. To top it all you can compile the CLR man !!!
Therefore it fills me with joy that Joel has the news that along with Ted (da ThoughtWorker) Neward they have re-worked the SSCLI referance book to match the SSCLI 2.0 release. Ted wrote the first draft and Joel brings his experince as a former PM on the CLR team to this edition and it is great to see. An early draft in full is available here. I'm pleased as punch about this as I used it along side books such as CLR via C# and Customizing the CLR.if you after more CLR books then I have an old (from 2003) list here.
More Posts
Next page »