ISerializable - Roy Osherove's Blog

We've just posted a preview of the next Isolator API coming out in august. It is AAA-style (Arrange, act, assert) and has many changes. We need your input so head on over, read it and tell us your thoughts. This is the time where you can make a huge difference. anything you don't like, say it in the comments.

If you are a reader of this blog and live in Italy (or hong kong)  - that means you live in a place where you can get the new iphone 3g with no sim lock. I have some money in my paypal account that I’d love to give you in exchange for you sending me the device (only Paypal). So, if you’re interested in getting one of them for me and shipping it to Israel, ping me at roy at osherove.com and let me know.

A while ago Udi Dahan wrote that he'd like a way to configure objects by just setting property values on an object. those "setters" will be intercepted and saved to  some config object (or container) for the future use of the application. Here is how simple it is to do with Isolator.

The registration for ALT.NET Israel is now open. You will need an OpenID (I use myopenid.com) to register. We are limited to 50 seats for the first event. First come first served. Worst case scenario you go into the waiting list.

Please only register if you are serious about attending.

The event takes place in two parts: Thursday eve. from 18:.30 to 20.30 and then the full day on friday (9-17.00). more details on the site.

I'm really happy to announce that we are going to have a small (30-70 people) ALT.NET Open Space event in Israel. The full details can be seen on Ken Egozi's blog. Ken is the one who drove Oren and myself into finally getting off our big butts and do this, but he's the one who find the nice location (SQLink offices) and we have Typemock to sponsor food and drinks for the conference.

Registration is not open yet. once the site is ready I will announce it. it is free on a first come first served basis and once we have all the spots filled up you will be put in a waiting list. It will take place on August 7-8 (3 weeks from now!) in Ramat Gan.

In the tradition of open spaces, the first half day (we meet in the evening of thursday) will be devoted to coming up with an agenda for the next day. the next (full day) will be totally run by the attendees. Should be lots of fun!

Here are the current results from the three questions I asked last week. Thanks to all those who answered. If you have not answered yet, it would be great if you took a minute to do so so we can all learn a little something.

What unit test framework do you use?

The (not so) big surprise is the MS Test is gaining and MvUnit is still way behind on usage compared to NUnit. When MS wants something, it gets it, I guess.

What mocking framework do you use?

Rhino Mocks is far ahead. It seems that, of the some 140 answers, almost the same amount of people don't do testing as there are those that use Moq, with Typemock Isolator, the only Commercial product in the bunch, coming in 3rd (not counting "none" obviously).  It sure is gaining, though. I would have expected more people to be using hand-made mocks and stubs, as that is a very simple way to get started, but that may be a sign that a lot of people may be abusing their mocking framework when they could be having much simpler tests.

What threading features do you use?

OK. 550 answer this so far. Far too many people are using Thread.Sleep which may not be the best course of action in many places. We will be using this in our upcoming Typemock Racer to decide which features to support at what order (since we are working incrementally with two week iterations).

I've gotten the chance to visit and consult various companies and I see this all the time: Over specification in tests, especially when doing mocks and stubs.

One main reason for that is that people use their stub as a mock object (stubs are fake objects that we will not assert on so that we can test something else). here is a small example from something posted on the alt.net mailing list:

[Test]

public void Can_Return_All_Plans()

{

IPlanDao mockPlanDao = MockRepository.GenerateMock<IPlanDao>();

mockPlanDao.Expect(x => x.AllPlans(_fakeAdmin)).Return(new List<Plan>{_fakePlan});

PlanController planController = new PlanControllerForTesting(_mockCommonDao, _mockCommonService, mockPlanDao);

ViewResult result = planController.All(false);

mockPlanDao.VerifyAllExpectations();

result.AssertViewResultNameAndViewDataType(typeof(PlanContainer), "Plans");

}

in this test the following line is used to "stub" out a return value into the system under test.

mockPlanDao.Expect(x => x.AllPlans(_fakeAdmin)).Return(new List<Plan>{_fakePlan});

no problem here. The problem lies in the last two lines of the test:

mockPlanDao.VerifyAllExpectations();

result.AssertViewResultNameAndViewDataType(typeof(PlanContainer), "Plans");

the second line (the assert) is probably the thing you really want to test. but the call to "verifyAllExpectations" is the one that makes the test very fragile. It is "asserting" that someone has actually called "GetAllPlans()" to get the results, when it clearly does not matter how one got the results. all that matters is that:

"Given a set of results returned into the application under test, the view in the controller is correct" (the translation of the last line in the test).

If by any chance the application later does several more calls to the stubbed out dao, or uses a different method to get results, the test could break, even though the application would still ultimately work with the same end result.

It's like me ordering a pizza delivery and then asking the delivery guy "did you get here by car or motorcycle?" . I realy don't care, as long as the end result: PIZZA, is correct.

So the general rules for me are:

• If you see both "Verify" and "Assert" in the same test, it is usualy a smell of over specified tests
• if you see "expect" and "Return" on a fake object, make sure that you name it "stubXX" or "mockXX" so that you can distinguish whether you want to call verify on it later or not(most cases should be "not")
• Try to test on the end result or end state rather than verify interactions.  The only time you absolutely have not choice but to test an interaction using verify is when calling void methods on external objects. that is clearly a one-way communication and is (or part of)the end result of what you are trying to accomplish.

A lot of people think that adding that extra "verify" just means it is a good thing since they are doing more assert. Well, they sure are testing more things, but they are internal things to the app's behavior and that is usually leading to brittle tests. Try to test on the end result or end state rather than verify interactions.

As we are working on the next set of features for Typemock Racer, the threading related test framework, I think it will be nice to know what features and thread constructs people use the most in their applications. The poll below uses flash, if you can't see it.

Please fill it in and we'll see what people use the most! While I'm here, I may as well ask you another couple of questions that need re-examining. Which mocking framework (if at all do you use?) and which unit test frameworks do you use?

Sasha and Alon have released an open source project called "Non Paged CLR Host" which has the following benefits(quoted):

1. Applications will benefit from no paging during normal operation.  Even if other applications are actively allocating memory, allocations performed under the non-paged CLR host will be locked into physical memory.
2. No paging will occur when the application is idle, providing a great benefit to low-latency processes such as GUI applications (even if the user has fallen asleep in front of the monitor).  The normal working set management scheme employed by Windows will not affect processes running under the non-paged CLR host.

That's Pretty cool. I've had the pleasure of working on a project with both of them, where CLR memory boundaries are constantly challenged. I wonder if it could benefit from such a piece of code!

PS

If you're not following sasha's blog - you really should.

Here's a sneak preview of the upcoming Typemock Racer product, which I'm currently working on. We should be out in private CTP in a couple of weeks I hope.

Typemock Racer tries to solve one huge problem for developers working in multi threaded environments: Detecting code that can result in deadlocks or race conditions. It does this by providing a framework API which you can use in your tests (under NUnit for example), that allows you to execrise your code under test until it finds a deadlock or times out.

Here's an example of how you'd use it. Consider the following class which has two methods, that if run by two seperate threads, can result in a deadlock condition:

At some point in time the threads could execute such that thread 1 is waiting on resource b, which is held by thread two , which is waiting on resource a, held by thread 1 - a pure deadlock.

Given this class we can write a test that looks like this:

The ThreadTest class has an "AddThreadAction" method that receives a delegate. you can use it to invoke your own code, it will be invoked in a different thread. IN this example we are creating two threads, each one running a different method on MyClassWithLocks.

We can just right click and run the test with TestDriven.NET to recieve the following results:

The output states that there is a deadlock that was found, with the exact steps that happened (with line locations) to reproduce this.

It currently only works with lock (monitor.Enter\Exit) but will support all locking constructs in .NET.

What if the methods you point to create their own threads? Like this:

Here is the output that you'd get if you run this test: