ISerializable - Roy Osherove's Blog

Unit Testing in C++ – it’s even harder than Inidian Pole Gymnastics

we’re adding more rules to Test Lint, these days, and one of them is called “Production Logic in Test”. It is quite different, and I think even more dangerous than the existing rule “Logic in Test” – because it talks about reproducing logic from within production code inside your unit tests.

What might that look like? here’s a simple example:

 1         [TestMethod]
 2         public void SomeTest()
 3         {
 4             string user = "user";
 5             ComponentUnderTest cut  = new ComponentUnderTest();
 6             string result = cut.SayHello(user);
 7 
 8             string expected = "hello " + user;
 9             Assert.AreEqual(expected,result);
10 
11         }

line 8 contains logic that is potentially reproducing logic under test. 99% certainty that the concatenation of the two strings might appear exactly like it does in the test, inside the production code.

so what’s the problem?

1. If the logic in production code contains a bug, and we are recreating the same logic in the test, we are essentially reproducing the bug as well – we are in fact expecting that bug to occur, so our test will pass because the bug is there.
2. even if we are not reproducing the bug from production, we might have our own bug in that piece of logic. Our test might fail, but because there is a bug in the expected test value – which we will not try to look for since we assume tests are correct (this would not be true if we were writing this test first!)

How is this rule different than just regular logic in test?

usual logic in tests usually tries to test several outcomes in the same test, or verify different things based on various conditions. a loop in a test will loop and may assert on several things, while an IF statement may exist so we only check something based on certain conditions.

in fact, logic in unit tests can appear in several places, and create the same problem as stated i #2 above:

• inside a unit test
• to avoid this, try to test the smallest thing possible that proves your point. for example, do you really need to loop over 10 rows in a memory table, or maybe just one row will do to prove something works or fails?
• inside a helper method (which also affects the readability of the test that calls that helper method)
• usually done for the same of reuse – this anti pattern in tests can both create bugs, and lack of readability or magic in the  calling tests. It’s magic if you have to ask yourself as you read the test questions such as “so where is my assert?” or “why am I sending in this string?”
• inside manual mocks and stubs (which also affects the readability of the test that calls that helper method)
• usually done for lack of knowledge that this is an anti pattern, people will create mocks and stubs that contains logic that might either reproduce some of the production code, or might make assumptions about what calling unit tests will send as parameters.
• this leads to lack of readability in the test
• magic strings or numbers in a test
• weird bugs
• many times that logic exists to accomodat ethe different needs of no more than a couple of unit tests that need different behavior from a stub
• it could easily be avoided by :
• creating stubs with hardcoded known behavior that is names in the type name of the stub, per each need of a specific unit test
• having the stubs take “orders” from the tests on how to behave – never to assume anything. for example.

as part of the last case study, I’ve also implemented another tool in our real world project – SmartInspect: a tracing tool. The developer had also given us this tool free of charge to do a case study with, which I had promised to publish on my blog (good or bad!). As a startup company we’re trying to conserve cash, so this is a great tradeoff – we need good tools, and we can pay with publicity if they are great! (hear that, JetBrains?)

Amazing tidbit: I reviewed SmartInspect last time during 2006, and it’s come a long way since then.

• EASY: it was relatively easy to get going with SmartInspect. The object model is quite nice, and I really liked the fact they have a special api for various common types (LogImage, LogException etc..). each type gets logs with its unique properties, and is visible in the very nicely done UI trace.:

The UI allows creating custom filters in your own tabs and panes, and is pretty powerful, but I’ve foudn its usability factor to be less than adequate. It needs more usability work, and I haven’t had time to contact their support to let them know (sorry about making you find out this way!). It also has some bugs (some “errors” are not shown even if you filter to see all error types..) but overall it has been very good.

The way we used it, is by using PostSharp aspects that weave custom tracing using SmartInspect in the the begin and end of each method. Otherwise you’d need to do it yourself. and I hate wasting my time that way. SmartInspect and PostSharp go together as nicely as burger and fris. Don’t leave home without them.

• Nice feature set. It does. It did everything I needed it to do, including support for “rolling” log files, that get recycled after a specific size, allowing encryption, and more. my favorite part of runtime view of logs as they happen inside the UI console. no need to look at the files after something happens. you can see it scroll by in real time.  it even supports logging from multiple thread and processes, and then seeing the log stack for each thread and process separately. I have not needed that feature yet, but future plans will force me to use it.

Summary:

I have not regretted choosing this tool, and plan to continue using it it other projects as well.

I’ve reviewed CryptoLicensing last year, and in the past few months, I’ve had the chance of implementing both CryptoLicensing and CryptoObfuscator for .NET in a real commercial project. the developer had given us free licenses to do a case study with it, and the results are very positive so far:

Crypto Licensing:

• EASY. Creating a licensing solution, including a key generation solution for CRM backend, checking for supported features and amount of runs for the program, and other very common tasks was extremely easy. it takes less than 10 minutes to integrate rudimentary licensing into a new application when you’ve done it before. If you’ve never done it it take about 30 minutes.
• SIMPLE. The code being using with the licensing is pretty simple and readable, and the object model is easy to work with. It has some interesting APIs that might confuse at first (differentiation between license keys and serials was a problem for me) but the documentation has been fixed to make things a bit more understandable.
• SUPPORT. the UI for the generator built in is not very intuitive. But it was MUCH less intuitive when I started working with it. During the past few months it has changes in many ways to alleviate various usability problems I’ve noticed in it, in many ways due to the quick response from the developer.
• Nice feature set. The licensing solution supports most very common cases in a very easy manner, and I pretty much could do whatever I wanted, except for one thing which I really missed – the ability to prevent multiple instances of the same application with same key in the local network (like resharper’s behavior). I’m told this is coming, but it could have come sooner. Other than that it supports activation remotely, trisal licensing, counting of runs, total time used, net time used, time since install and many other more advanced features. Pretty cool for such a small footprint.

Crypto Obfuscator:

• RELATIVELY EASY. It was quite easy to get it to work with one assembly, but once we got into multiple assemblies with dependencies we’ve hit some bugs. Those were fixed very quickly and it seems to work great with a very diver solution of dependent projects.
• SIMPLE: making it work as part of the build was very simple indeed. changing an option on it is also simple. the way everything should be in software.
• SUPPORT: just as good (same developer). I had various usability problems with it that were fixed quickly.
• Nice feature set: it has everything I needed (including ability to prevent Reflector from seeing various bits of source code).

Summary:

I’m really happy I chose these tools. They have, still, some usability issues. but they work great, and they saved us some real time when we needed to focus on actual business value instead of dealing with crap like this. That’s about as positive as it gets in component world – get a component so it can get out of your way when you develop the really important stuff. As a result we’re now looking into using Crypto licensing for MFC for our Isolator for C++ product line. I’ll let you know how that goes!

another TDD masterclass is done, and this time it was Munich! Lovely people, and smart programmers :)

made a few changes this time – had an excersice about doing automated builds and CI, removed rhino mocks and added a bunch of other frameworks (FakeItEasy, JustMock and others) and various other small things.

at the last day of the course, I asked each person on the most important thing they learned, and something that surprised them during the course.

here’s the list we made:

•How to practice TDD

•Isolation frameworks abilities, differences and categories

•Difference between test and production code standards

•Pay attention to requirements

•TDD is doable, also in legacy

•Simple tests

•Legacy code refactoring is hard

•Better unit tests

•It’s not hype

•Reviewing is easier than thought

•Pair programming is productive

•How to refactor and test legacy

•Power of manual fakes

•Tdd makes testable code

•Tests as specs

•How to write fast tests, productively with tdd

•RTM tests are harder than you think

•Spec and Interface based development

•Code gets written differently

•Incremental work, simply

•Lots of different ways to do the same concept

•How to handle statics

•Different people are different

•Brownfield is tough to tdd

There is a new version of Test Lint – a test review tool for visual studio. It will check your unit tests for common problems and teach you on making your tests more readable, maintainable and trustworthy.

The addin is still free and has a new rule (about using DateTime). but it also comes with a command line runner, that allows you to run test lint as part of your automated build. you just provide it with an /sln=mysln.cln parameter and go.

The command line runner will not be free. We will be coming out with a pricing plan for it sometime this week. For now, feel free to give it a try. I’d love to hear your comments!

- Added: *Command Line Runner* Tool
    ** Limited Preview (TestLintCMD.exe)
    ** for automated builds. Trial Version allows 15 initial runs.
    ** parameters are:
    ** TestLintCmd.exe
    **            /sln="[solution file]"
    **            /license="[optional. path to license file]"
- Added: Test Lint will ignore projects with 'integration' in their names
- Added: new rule: DateTime.Now

- Added: link to submit a bug from within test lint popups
- Added: missing tooltip on lint icons
- Added background statistics on Test Issues

- Fixed bug: Missing asserts did not detect
    X.foo().Should...()   [fluent methods)

We’re happy to bring you the first episode of the new and revamped “This week in testing” with Roy osherove and Ariel Raunstien.

The episode is 32 minutes long.

You can watch it in full at our new site.

I’ve been Editing lots of videos lately. My company got a video camera: Canon Legria FS200. It saves the movies in a digital format – as MOD files.

Unfortunately, Adobe Premiere doesn’t work with these files. I needed software to convert MOD files to QuickTime or mpeg files. I found a good free one : It’s called Mpeg StreamClip:  It works well. and it’s pretty fast. And it’s Free.

What’s not to like?

I got Ingo Rammer’s permission to post this video of one of his Teched EMEA 2008 talks – it’s about hardcore production debugging, and it’s a wonderful talk. I highly recommend you watch it. Also you should consider getting the book Advanced .NET Debugging – it’s wonderful (though pretty advanced!)

FYI, you can find many other .NET and unit testing videos here.

We are looking for a .NET\C++ developer to join the growing Typemock ranks. You need to:

• Live in Israel
• know .NET very well (at least 3 years .NET experience – VB.NET or C#, and willing to learn the other one)
• Have some C++ experience (recent – sometime in the past couple of years)
• Be interested in Agile development, unit testing and TDD (you don’t have to be an expert. You’ll become one on the job.)
• have very good english
• PASSION for programming
• Advantage to C++ hardcore devs but you don’t have to be one
• Advantage to Open source contributors   but you don’t have to be one
• Advantage to public figures (bloggers, speakers..) but you don’t have to be one

You will be working on one of our products, or several of them along the way. Including Typemock Isolator, Test Lint, TeamMate and future products we are working on! We are counting on all our developers to be part of the design process, to take active part in support and customer meetings, and the first day of every two weeks is dedicated to pet projects – you work on anything you want (even if it’s not to do with Typemock)!

send an email with your ENGLISH CV to royo AT typemock.com