A Portion of Buff

Visual Studio.Net 2001

On the last project I was on, there were a lot of ex-Java guys who were used to working with IntelliJ and Eclipse, and who derived endless pleasure from ridiculing Visual Studio's comparative shortcomings.  They renamed it 'Visual Retardio' ('Visual Subbuteo' came a close second).  When Jetbrains started the ReSharper EAP earlier this year we all jumped on board in a flash, and stuck through months of pain and suffering (it was nicknamed 'ReBlunter' for a time) until finally the finished product was released.

Then Visual Studio 2005 went beta, and I was hoping that Microsoft would now have an IDE that was competetive with the best Java IDEs, and that would give us all the things we had become used to with ReSharper.

After using VS.Net 2005 for a few weeks, and having used VS.Net 2003 with ReSharper for the last six months or so, I have come to realise the following:

I much prefer VS.Net 2003 + ReSharper over VS.Net 2005, for the following reasons:

1) Refactoring

• ReSharper's refactoring menu is context sensitive, and will only present those which are relevant to the current cursor location.  2005 just gives you all of them, then throws an error when you try and use one in the wrong place.
• ReSharper has Introduce variable.  I use this all the time.
• ReSharper has a sweet inline rename for local variables and parameters, and will even suggest names for you based on the type.

2) Highlighting

• ReSharper will colour fields, local variables and methods.
• It will colour unused or unreachable code.
• It will highlight all usages of a field/variable/method within a file.
• Type this into any method in 2005:

      Console.WroteLine();

What happens?  Very little.  Try it with ReSharper and it will colour 'WroteLine' in red, and present you with a drop-down list of all the valid Console methods (it will even narrow the list to the best matches).

4) Code Navigation

• 2005 now has a usable Goto Definition, but you can't navigate to sub or parent types.
• 2005 now has Find References, but it doesn't find references.  It finds symbols matching the currently selected element, which is a very different thing.  If I'm looking for references of a method, I don't want to be presented with a list of similar-looking methods on other classes, and I don't want the method I just selected to be present in that list.  It's just broken.
• Guess what?  Both these features really work in ReSharper.  And it has decent 'Find Type' and 'Find Method' feature, which narrows the list as you type.

5) Quick Fixes / Smart Tags

• I'm no big-city usability expert <gasp!>, but to me, Smart Tags have a big failing: they require three steps to select an option via the keyboard.  First, you use a hot-key to get the menu up.  Then, you move the cursor down to the option you want, and then you hit return.  This wouldn't be so bad, but if there's only one option in the list, you still have to navigate to it!  ReSharper's Quick Fixes automatically select the first or only option in the list, and in some cases (if you use a type without importing it with a 'using' statement, for example) it will bring up a tooltip with the relevant option, requiring only one key press to activate it.

6) Intellisense

• ReSharper will give you a list of types which haven't yet been imported.  When you select one, it will complete your code and add the 'using' statement for you.  It will also auto-complete variable assignments, including brackets and a semi-colon if required.

7) 'Miscellaneous'

• ReSharper's 'Extend Selection' feature is quite hard to explain, but very useful.  Essentially it's like ctrl-click, but it selects outwards from the current cursor position, and is scope-aware (eg it will select the name of a method call, then the call + its parameters, then anything enclosing that, then anything enclosing that, right up to the file level).
• Duplicate Line - like [ctrl-c, cursor down, ctrl-v], but all in one go.

I really could go on, but this sounds too much like an advertisement already.  Suffice to say that I am disappointed by the first beta of 2005, and I hope that Microsoft can improve things a little by the time they release.  I am looking forward to the day when Visual Studio really understands code, rather than being a glorified text editor.

Double whammy

Anyone who has passed a managed delegate as a parameter to an unmanaged function will be familiar with the oh-dear-the-GC-just-pulled-the-rug-from-under-me NullReferenceException you get if you don't keep save a reference to that delegate.  It's hard to debug, but once you are aware of the problem you tend not to make the same mistake again.  Or. So. I. Thought.

So I'm merrily hacking away with .Net 2.0, trying out the new flashy bits like generics and anonymous methods, when I tried out the new type-inference features in C#.  Essentially this means that C# creates the delegate for you, and code like this:

button.Click += new ClickEventHandler(OnClick);

can be simplified like this:

button.Click += OnClick;

which is much nicer.  It also means you can pass delegates to, you guessed it, unmanaged code without explicitly declaring a new delegate instance, and then spend a happy few hours debugging a stack-less NullReferenceException, safe in the knowledge that the problem can't be related to the delegate being garbage collected, because you'd never make that mistake again.  In my defence, I was clearly drunk on syntactic sugar.

Which leads me to the second whammy: in 2.0, you don't always get a NullReferenceException in this situation.  Sometimes, you get an AccessViolationException, which is a new type meant to help distinguish between null references and other types of AVs.  Unfortunately, the CLR seems to be inconsistent about which one it chooses to throw (I was getting NRE on some runs and AVE on others) and led me to investigate the wrong flavour of bug.

Oh yes, we're fully XML enabled

I missed one off the Programming by Superstition list:

Myth:

DataSets store their data internally as XML.

Rationale:

Microsoft said so!

When ADO.Net was announced, there was a lot of marketing guff about how it was "based around" XML, and how XML was a "first-class citizen".  After all, it was the year 2000, and no CxO worth their salt was going to buy into a solution that wasn't "fully XML enabled".  In fact, there is at least one article on MSDN which refers to DataSets being "natively an XML structure", so perhaps Microsoft believed their own hype.

Truth:

DataSets aren't really the key structure here - they don't do much more than group DataTables together.  In turn, DataTables group DataColumns together, and DataRows act as a "horizontal slice" into a group of DataColumns.  Going one level further, a DataColumn is just a wrapper around one of a number of classes inheriting from DataStorage (BooleanStorage, ByteStorage, CharStorage and so on), and those are just strongly-typed arrays with a bit of collection-management logic thrown on top of them.  So, no XML.  Sorry.

(And just for the record, they would have been feckin stupid to have used XML anyway).

Programming by superstition

Myths and half-truths around .Net and C# best-practices crop up frequently in discussion forums, with rationales ranging from utterly believable to indefensible.  I thought I would catalogue the ones I see most frequently.  I realise I am leaving myself open to ridicule by prefixing my 'corrections' with 'Truth', but someone has to make a stand :-)  Feel free to correct me.

Myth:

Always set your objects to null (or Nothing) when you're done with them.

Rationale:

The garbage collector won't be able to free an object if any another object is referencing it. 

Truth:

The garbage collector can free any object that is unreachable from any of the various heap roots.  If object A references object B, and object A is unreachable, object B can be released if it has no other references from reachable objects.

Myth:

StringBuilder.Append() is always faster than String.Concat().

Rationale:

String operations always create new string objects, whereas StringBuilder uses internal magic to avoid expensive allocations.

Truth:

Using a StringBuilder is sometimes (even usually) faster than using simple string concatenation, but there is a performance cutover point.  Various people have found this to be around the 5 - 10 concatenations mark.  The bottom line being that replacing this:

string fullName = firstName + " " + lastName;

with this:

StringBuilder builder = new StringBuilder();

builder.Append(firstName);

builder.Append(" ");

builder.Append(lastName);

string fullName = builder.ToString();

Myth:

Strings are passed to, and returned from, methods by-value.

Rationale:

Strings are immutable, therefore they must have value type semantics, therefore must be copied by value.

Truth:

Strings have certain value-type semantics (namely, immutability), but are otherwise heap-based reference types, and references to them can indeed be passed or copied.  Passing a string reference to or from a method does not copy the underlying string.

Myth:

Dispose() releases an object's memory.

Rationale:

Well, it's called 'Dispose' isn't it?

Truth:

Dispose() is merely a convenient place for a class' developer to place any resource clean-up code, in the hope that clients will call it.  Dispose does not free the managed memory allocated for an object.  The only thing that can do that is the garbage collector; there is no way to deterministically (or manually) release memory.

Myth:

Keeping ADO.Net connection objects open for the duration of a [Page/Control/1000-line method]'s lifetime is more efficient than repeatedly opening and closing it around each operation.

Rationale:

Connecting to a database is expensive, so try to avoid doing it too much.

Truth:

If your managed provider allows connection pooling, and if pooling is enabled (it is, by default, for the managed SQL Server provider at least), calling Close() on the connection object does not simply close the underlying connection.  It merely releases that connection back to the pool where it can be associated with another managed connection object at some time in the future.  This is a Good Thing, especially in multi-threaded environments such as ASP.Net, because physical connections are rare resources and should not be coveted.  (By the same token, calling Open() on a managed connection when a pooled physical connection is available will simply associate the former with the latter).

By keeping a managed connection in an open state, you are preventing objects in other threads from getting hold of them, leading to potential bottlenecks.

Myth:

Int32 and int are not the same thing (ditto float/Short, long/Int64, string/String, etc.).

Rationale:

They're spelled differently.

Truth:

The C# simple type keywords are interchangeable with the equivalent BCL types; they are aliases.

The remarkable staying-power of of System.Assembly

There was an interesting bug in NMock until recently (it's been fixed) which saw memory usage during unit test runs increase in relation to the total number of mock objects created during the run.  The culprit was .Net's inability to unload assemblies from a running AppDomain: each mock was being hosted in a brand new dynamic assembly - which never gets garbage collected - so you could easily end up with a few hundred assemblies languishing in the root domain, eating memory.  We fixed it by creating one assembly for all mock types, and by caching identical types for the duration of the test run.

Today, another bug was raised against NMock, which involved calling GetManifestResourceNames() on all loaded assemblies.  Dynamic assembliles will throw NotSupportedException for this method (and for a couple of others), so if any code which relies on this call is used alongside NMock (in this case it was NVelocity), we break it.

I looked at a couple of workarounds, including saving our mock assembly to disk and reloading it as a static assembly so it could play nicely, but noticed that AssemblyBuilder, which we use to generate the mock assembly, is never garbage collected either!  So, kids, be aware that if you generate assemblies at runtime, even just to save them to disk, your AssemblyBuilders will hang around for ever, holding you memory hostage and calling you "poo-breath" behind your back.  The solution, of course, is to do your code-gen in a separate AppDomain, which is pretty weak but at least you can throw it away when you're done.

Does anyone have a good reason why assemblies cannot be garbage collected?

Build, old woman! Build like the wind!

What's your build speed?  How long does it take between you clicking on the check-in button and the green (or red!) light appearing on the task bar telling you your build succeeded (or failed!).  In our case a finished build is announced by Homer shouting "woohoo!" or "D'oh!", and it takes around three minutes for the "express" build - just compilation and unit tests- and twenty minutes for the full build, which includes acceptance tests, MSI compilation and deployment.  That's for a 200,000 line application, which I suppose is medium-sized.

Colleagues on different projects have remarked on this as being pretty quick, and have noted build speeds of 1 - 2 hours for similar sized applications.  The biggest overhead for them seems to be database activity, which we try to minimise by using mock objects and abstraction layers like Neo.  We also try to keep compile times as short as possible by aggresively deleting dead code, useless comments and using statements etc., and to keep unit tests small and well-targeted.

So, it would be interesting to get a sense of how long other people sit around for waiting for a build, and what the biggest contributing factors to slow build speeds are.  Does your build take so long you only run it at night?  Do you find it unnecessary to build more than once a day (or even less frequently?).  Let me know.

Jetbrains 'ReSharper'

Jetbrains briefly posted a feature list for the first beta of their VS.Net plugin (hopefully out this week), and then pulled it. Watch me get in trouble for posting it here (excuse the lack of formatting):

Do you know what your constructor is doing?

Earlier, Peter Torr posted about the differences between language-enforced rules and framework-enforced rules (calling class constructors, specifically).  An interesting and related thing I discovered a couple of weeks ago was that constructors are not guaranteed to be called at all by the framework, let alone twice.  Again, it involves emitting MSIL (or not, as we shall see), rather than C#, but it's still a useful/dangerous/so-what feature.

Here's the MSIL that CSC emits by default for an empty constructor:

ldarg.0
call       instance void [mscorlib]System.Object::.ctor()
ret

As you can see, the base constructor is automatically called.  This call, as CSC would have it, happens before anything else in the current constructor is executed.  The trick - you guessed it - is simply to omit this call.  A constructor then becomes:

ret

Marvellous.  And what's the big deal?  Well, it turns out to be a solution to a limitation I keep running into with NMock, namely that it cannot mock a class without a parameterless constructor (NMock creates testable sub-classes on the fly using Reflection.Emit).  I got so fed up of creating single-implementation interfaces or empty constructors just to mock a class that I decided to investigate, and that's what I came up with.  If constructing the base class is a chore (because of the parameters it takes), then don't call it.  If it doesn't have an empty constructor, so what?  I don't have to call it.

You do have to be a bit careful calling unmocked methods who rely on state that should have been initialised by the constructor, but in certain situations it makes life much much easier.

The code should make it into NMock in the next couple of weeks, if I can get my arse in gear.

Fowler on Inversion of Control

Like falling off a blog

I've been busy working on a good-sized project for Thoughtworks over the last 9 months, which is one reason for the lack of posts.  The other is that it's taken me about 9 months to think of something to write about.  Here's something I thought worthy of note:

Neo is an open-source object/relational mapping tool (I'll probably get in trouble about calling it an O/R mapper by its author - he calls it an "object facade") developed by my colleague Erik Doernenburg then tweaked, debugged, used and abused on the aforementioned project.  At least a couple of other teams are using it now, and it's very close to a 1.0 release.

Neo works by generating both .Net classes and database tables from an XML schema. It then creates a one-to-one mapping between the two (which is why I should really call it a facade) using ADO.Net and provides typed collections, factories and query mechanisms for working with the generated classes.  The resulting object model is remarkably simple to use and has saved us from much of the hell of manual persistance.  Let's take a look at an example:

IDataStore store = new SqlDataStore(connectionString);
ObjectContext context = new ObjectContext(store);
Title title = new TitleFactory(context).CreateObject("TC1234");
title.Price = 12.99M;
title.Publisher = new PublisherFactory(context).FindSingle("Name = 'New Moon Books'");
context.SaveChages();

In order of appearance:

• IDataStore - Provides a means for an ObjectContext to store and load DataTables.
• ObjectContext - Provides a scope for objects to exist within, performs bulk load/save operations and helps manages the lifecycle of loaded objects.  It is backed by a DataSet which hooks all the entity tables together.
• TitleFactory - Used for creation and location of objects.  Each entity class has its own strongly typed factory.
• Title - An example of a Neo-generated entity class.  It is backed by a single DataTable (each instance represents one row in that table).
• Price/Publisher - Two properties representing both a column entry and another Neo object.
  

Neo, like ADO.Net, doesn't care where its data originates - we use SQL Server, local XML files and web services (using DataSets makes it easy to hook it up to a web service).  Working in-memory also makes testing much easier and quicker.  Our app was in development for about 4 months before it even saw a real database (which was probably too long in hindsight, but that's a different story).

There are limitations, of course:

• There is a one-to-one relationship between classes and datatables, which can make complex systems hard to model correctly. 
• The only supported database at the moment is SQL Server, although it would be trivial to implement IDataStore for others.
• Currently, Neo is really only suited to projects which have full control over their database schema.

Anyway, go and check it out, and let us know what you think.  I can post more information and samples if requested.