A Portion of Buff

Noise 0.3 released

2005-04-11T23:31:00Z

I've put a new cut of Noise, the premier solution for C# VST developers (ahem), up on GForge. The changelog:

* Finished implementing v1.0 of the SDK _apart from_ setChunk()/getChunk() because I don't know how best to handle them yet. It would
be nice to do something a bit smarter than a byte stream. Suggestions welcome.

* Refactored ManagedBridge a little bit so it directly returns values instead of using out params.

* Added very basic error handling to HostPlugin - if your plugin fails to initialise properly, it will report nicely back to the host.

* Added an example of a compressor. Doesn't work properly yet, but it _sort of_ sounds compressor-like.

* You can now provide your own unique ID to the host (see CompressorPlugin for an example).

* Made some of the process classes mono, which makes them a bit easier to understand.

The subdued garbage collector

2005-04-05T23:41:00Z

I want Noise to support any .Net language, not just C# (although why anyone would use another language...). At the moment, the plugin interface exposes unsafe methods - Process and ProcessReplacing - which take float* parameters and are therefore unusable by a pussy language like VB.Net. Pointers are used because it allows us to write directly to the host's memory instead of allocating our own buffers and copying them back to the host.

If we are prepared to take a performance hit, we can make this interface safe. Not only is there the cost of copying unmanaged memory to a managed buffer on the way in, and then back again on the way out, we need to allocate some managed memory in the first place. Allocations lead to collections, collections lead to hate, hate leads to suffering.

However, we can take advantage of the fact that the maximum number of samples being passed to the plugin rarely changes (usually not until the host's latency setting has been changed, which is hardly ever under normal use), although the host is free to send fewer samples if it wishes. The host tells us the maximum block size by calling the SetBlockSize() method, which is a good place to allocate our buffers:

public override void SetBlockSize(int blockSize)
{
    leftBufferIn = new float[blockSize];
    rightBufferIn = new float[blockSize];
    leftBufferOut = new float[blockSize];
    rightBufferOut = new float[blockSize];
}

and in the Process or ProcessReplacing methods, we can just fill these buffers using the Marshal class:

public override void ProcessReplacing(float* inLeft, float* inRight, float* outLeft, float* outRight, int sampleFrames)
{
    Marshal.Copy(new IntPtr(inLeft), leftBufferIn, 0, sampleFrames);
    Marshal.Copy(new IntPtr(inRight), rightBufferIn, 0, sampleFrames);
    Marshal.Copy(new IntPtr(outLeft), leftBufferOut, 0, sampleFrames);
    Marshal.Copy(new IntPtr(outRight), rightBufferOut, 0, sampleFrames);
//...
}

It doesn't matter if sampleFrames is less than blockSize, we just copy that many samples to our buffers. This enables us to call out to a safe method which deals with float arrays, and performance is constant because no new allocations occur. Hurrah.

Noise 0.2 source released

2005-03-25T14:58:00Z

Now installs to the GAC (this was the only way I could let hosts load a plugin from an arbitrary location. Sometimes assembly-resolution sucks). Added a reverb plugin to the package, and generally fixed up the solution. To get up and running:

1) Download and extract noise-source-0.2.zip somewhere.
2) Download the VST Plugin SDK from http://www.steinberg.net/Steinberg/Developers.asp
3) Copy everything from vstsdk2.3\source\common into Noise\HostPlugin\vst
4) Build the solution
5) Open project properties for Noise.ManagedVst, go to Configuration Properties-->Debugging
6) Change Debug Mode to "Program"
7) Change "Start Application" to point to VstHost.exe (under Noise\ by default)
8) Hit [ctrl +] F5
9) Load up your favourite synth plugin, then chain HostPlugin.dll after it (deployed in Noise\build).
10) Enjoy

On microbenchmarks and red faces

2005-03-23T13:49:00Z

Dear VB.Net H8t3R,

Try making a single call to Strings.Trim(s) before you start benchmarking. I think a static constructor just bit you on the ass.

Love,

Your nemesis

Keeping it real-time (3)

2005-03-22T00:03:00Z

These late nights are going to catch up with me, but until then...

I've kicked off Noise, an open source project for this managed VST plugin thing I'm working on:

(Last year I was working on an abortive software synthesizer called "Noise!", so it seemed appropriate to recycle the name).

It's obviously quite rough at the moment, but please, if you use any VST-compatible audio software (Cubase, SoundForge, FruityLoops etc), download it, give it a try, and let me know if you had any problems (positive feedback is nice too). If you don't have a VST host, there's a free one included, so you can still play around if you're interested.

So far I've got delay, lo/hi-pass filter, gain and reverb plug-ins - the latter being the most fun to do and the most difficult to get my head around. One thing that struck me tonight was just how incredibly fast computers are. I know I should be accustomed to that, being a developer, but when you're writing code which applies 12 different filters to 44,100 stereo samples per second, in parallel to a software synthesizer which is generating those 44,100 stereo samples per second with its own oscillators, filters and envelopes, on top of a host application routing all these samples from the synth, through your plugin (via layers of C, Managed C++ and C#) and out to a sound card, and it only takes 3% of available CPU time, it kind of brings it home.

Here's a stereo comb filter:

public class CombFilter
{
    float feedback;

    float damp1;
    float damp2;

    float lastLeftValue;
    float lastRightValue;

    int leftIndex;
    int rightIndex;

    float[] leftBuffer;
    float[] rightBuffer;

    public CombFilter(int leftTuning, int rightTuning, float damp, float feedback)
    {
        this.feedback = feedback;
        this.damp1 = damp;
        this.damp2 = (1 - damp);
        this.leftBuffer = new float[leftTuning];
        this.rightBuffer = new float[rightTuning];
    }

    public float ProcessLeft(float input)
    {
        lastLeftValue = (leftBuffer[leftIndex] * damp2) + (lastLeftValue * damp1);

        leftBuffer[leftIndex] = input + (lastLeftValue * feedback);

        if (++leftIndex >= leftBuffer.Length)
        {
            leftIndex = 0;
        }

        return leftBuffer[leftIndex];
    }

    public float ProcessRight(float input)
    {
        lastRightValue = (rightBuffer[rightIndex] * damp2) + (lastRightValue * damp1);

        rightBuffer[rightIndex] = input + (lastRightValue * feedback);

        if (++rightIndex >= rightBuffer.Length)
        {
            rightIndex = 0;
        }

        return rightBuffer[rightIndex];
    }
}

Keeping it real-time (2)

2005-03-15T23:31:00Z

I had a request for more (more!!!), so I'll explain how some more of the managed plugin system works. At the root of it, we have the VST SDK-derived class. This is a standard C++ class which inherits from AudioEffectX (defined as part of the SDK). Any VST host will expect an instance of this type, and will call certain methods to both interrogate your plugin about its capabilities and pass it the magical float** inputs and outputs. (I remember reading somewhere about three star C programmers, and how they were a breed unto themselves. We only have two stars here, but as .Net programmers, I think we should allow ourselves a smug grin).

As Managed C++ classes cannot inherit from unmanaged classes, our root plugin must be unmanaged. However, unmanaged classes can have references to managed classes, via the gcroot<> template (there may be other ways - I have not discovered them yet). A VST host will typically want to know how many inputs and outputs a plugin can handle, whether it can be a parallel ("send") or serial ("insert") effect, whether it has its own GUI for editing parameters, etc. My goal is to delegate as much as possible to C#, so most of these calls must be marshalled directly to managed classes. The good news is that MC++ makes this marshalling really fricken easy. Anyone who's done much P/Invoke will know how fiddly all those MarshalAs attributes can be to get right. Well, most of the time, with MC++, It Just Works. You have to make a few decisions about which side owns which piece of memory, but I'm 90% convinced that doing interop from the unmanaged side is just easier. Although maybe that's just the pointers talking. An example:

void VstToManaged::process (float **inputs, float **outputs, long sampleFrames)
{
    m_pPluginBridge->Process(inputs, outputs, sampleFrames);
}

That takes us from an unmanaged, VST 0wn3d method to MC++. No mucking about with DllImport, just include the header file.

Did I just say that?

Anyway, taking the process() method further, I decided that most C# weenies wouldn't know what to do with two levels of indirection, so I dereferenced the left and right channels and created an interface thusly:

public interface IVstPlugin
{
    unsafe void Process(float* inLeft, float* inRight, float* outLeft, float* outRight, int sampleFrames);
}

There's more on this interface than that, but I'll keep it simple for now. The code from an MC++ point of view?

void ManagedBridge::Process(float **inputs, float **outputs, long sampleFrames)
{
    float *in1 = inputs[0];
    float *in2 = inputs[1];
    float *out1 = outputs[0];
    float *out2 = outputs[1];

    plugin->Process(in1, in2, out1, out2, sampleFrames);
}

Interop schminterop. And, as I'm feeling philanthropic, here's a mono Biquad filter in C#. Feed it a stream of samples and see what comes out:

using System;
namespace ManagedVst.Processes
{
    public enum FilterType : byte
    {
        LPF,
        HPF,
    }

    //Ported and adapted from Tom St Denis' C version (http://www.musicdsp.org/showone.php?id=64)
    public struct BiQuad
    {
        //Coefficients. Don't ask me what they do.
        double A0, A1, A2, A3, A4;

        //X1 is the input from the previous step, X2 is the input before that
        //Y1 is the output from the previous step, Y2 is the output before that
        double X1, X2, Y1, Y2;

        public void Init(FilterType filterType, double sampleRate, double frequency, double bandwidth)
        {
            //A frequency of 0 can lead to the filter producing
            //very small (denormal) numbers and NaNs. This is bad because
            //most processors are very slow at calculating very small floating point numbers, so we should
            //clamp such values to something larger. This might be slightly superstitious, but including
            //the following check banished 100% CPU usage spikes. Alternatives welcome (double.Epsilon didn't
            //appear to work).
            if(frequency == 0d)
            {
                frequency = 0.00000000000001d;
            }

            const double LogE2 = 0.69314718055994530942;

            double omega = 2 * Math.PI * frequency / sampleRate;
            double sinOmega = Math.Sin(omega);
            double cosOmega = Math.Cos(omega);
            double alpha = sinOmega * Math.Sinh(LogE2 / 2 * bandwidth * omega / sinOmega);

            double a0 = 1 + alpha;
            double a1 = -2 * cosOmega;
            double a2 = 1 - alpha;

            double b0 = 0;
            double b1 = 0;
            double b2 = 0;

            if (filterType == FilterType.LPF)
            {
                b0 = (1 - cosOmega) / 2;
                b1 = 1 - cosOmega;
                b2 = (1 - cosOmega) / 2;
            }
            else //filterType == FilterType.HPF
            {
                b0 = (1 + cosOmega) / 2;
                b1 = -(1 + cosOmega);
                b2 = (1 + cosOmega) / 2;
            }

            A0 = b0 / a0;
            A1 = b1 / a0;
            A2 = b2 / a0;
            A3 = a1 / a0;
            A4 = a2 / a0;
        }

        public float Step(float input)
        {
            double output = A0 * input + A1 * X1 + A2 * X2 - A3 * Y1 - A4 * Y2;

            X2 = X1;
            X1 = input;

            Y2 = Y1;
            Y1 = output;

            return (float)output;
        }
    }
}

Keeping it real-time

2005-03-15T00:06:00Z

Since we last spoke, I have realised that a managed VST plug-in is a far from practical application for hosting the CLR. Which means no more COM haxx0ring. Shame.

On the upside, I have discovered the twin joys of Managed C++ and porting DSP code from C to C#. MC++, after several failed attempts, now shuttles calls from the VST host through to C#. The CLR is of course loaded automatically whenever the managed DLL is touched, so hosting it explicitly is pointless.

It is splendid fun. At the moment, much of the real-time processing is done with unsafe code (the VST host passes in pointers to the input and output buffers), but it’s nicer than working in C. Here’s a simple delay:

public class Delay
{
      float samplingRate;
      float[] delayLine;
      int delayLineIndex;
      int bufferLength;
      private float delayTime;
      private float feedback;
      private float dryLevel;
      private float wetLevel;
      private const int MAX_DELAY_TIME = 1000;

      public Delay(float delayTime, float feedBack, float dryLevel, float wetLevel, float samplingRate)
      {
            this.delayTime = delayTime;
            this.feedback = feedBack;
            this.dryLevel = dryLevel;
            this.wetLevel = wetLevel;
            this.samplingRate = samplingRate;

            int maxBufferLength = CalculateBufferLength(MAX_DELAY_TIME);
            delayLine = new float[maxBufferLength];
            bufferLength = CalculateBufferLength(delayTime);
            this.delayLineIndex = 0;
      }

      unsafe public void Process(float* input, float* output, int sampleFrames)
      {
            for (int i = 0; i < sampleFrames; i++)
            {
                  output[i] += (dryLevel * input[i]) + (wetLevel * delayLine[delayLineIndex]);
                  delayLine[delayLineIndex] = input[i] + (feedback * delayLine[delayLineIndex]);

                  if (++delayLineIndex >= bufferLength)
                  {
                        delayLineIndex = 0;
                  }
            }
      }

      public float DelayTime
    {
            get { return delayTime; }
            set
            {
                  this.delayTime = value;
                  this.bufferLength = CalculateBufferLength(value);
            }
      }

      public float Feedback
      {
            get { return feedback; }
            set { feedback = value; }
      }

      public float DryLevel
      {
            get { return dryLevel; }
            set { dryLevel = value; }
      }

      public float WetLevel
      {
            get { return wetLevel; }
            set { wetLevel = value; }
      }

      private int CalculateBufferLength(float delayTime)
      {
            return (int)(delayTime * samplingRate / 1000);
      }
}

Performance hasn’t been an issue so far, but I haven’t tried to do anything particularly taxing. The real test will come when I start creating garbage, although I think Gen 0 collections will be tolerable.

Practical applications for CLR hosting

2005-02-21T18:05:00Z

Well then. It's been a long time since I last got cozy with the unmanaged side of the CLR, but all the interop I have been doing lately has made me cocky. This post is a write-up of my attempts to develop a VST plugin in C#.

A what?

VST stands for Virtual Studio Technology. It's a standard developed by Steinberg for producing audio software components. The idea is that a VST host application (Cubase, Logic Audio, ACID etc) acts as a software mixing desk, routing audio in and out of these plugins. A plugin can be an effect, such as reverb or chorus, which changes the sound fed into them in some way; or an instrument which produces sounds of its own.

Steinberg provides an SDK which developers can use to make plugins but it only provides a C++ API. This is understandable as plugins need to be fast and work in realtime, and any long pauses could cause glitches in an audio stream or throw timing off. In addition, C++ is the de facto standard for writing audio software. On the other hand, as we have seen with Managed DirectX, it is possible to write performance-sensitive software in managed code as long as an effort is made to reduce the impact of garbage-collection and other overhead.

VST plugins are loaded into the host application as native DLLs, so I needed some way to call out into managed code from that point. There are a few options, but I took the route of hosting the CLR and calling into managed code with COM interop: partly because it sounds cool, and partly because it was the first thing that worked.

I’m not going to explain how to use the VST SDK – there are plenty of tutorials out there (this is an excellent one). This post is about hosting the CLR and calling into managed code, and assumes you can stumble your way through C++ COM code.

Plugins inherit from a base class provided in the SDK called AudioEffectX. The constructor would seem to be an appropriate place to start up the CLR, because it is called before any performance-critical processing takes place. The code to do this is well-documented (here, for starters), but looks something like this:

ICorRuntimeHost *pRuntimeHost = NULL;

CorBindToRuntimeEx(

NULL,

CLSID_CorRuntimeHost,

IID_ICorRuntimeHost,

(LPVOID*) &pRuntimeHost);

pRuntimeHost->Start();

On its own, this won’t get us very far. We need some managed code to run! Getting a reference to the default AppDomain is easy enough:

IUnknown *punk = NULL;

pRuntimeHost->GetDefaultDomain(&punk);

_AppDomain *pDefaultDomain = NULL;

punk->QueryInterface(__uuidof(_AppDomain), (PVOID*) &pDefaultDomain);

The _AppDomain interface is, essentially, identical to the managed AppDomain class which means we use it to load assemblies, create objects, create new AppDomains and all that good stuff. My managed code is in an assembly called “ManagedVst”, and I want to create an object of type “PluginAdapter” in the “ManagedVst” namespace:

_ObjectHandle *pObjHandle = NULL;

pDefaultDomain->CreateInstance(_bstr_t("ManagedVst"), _bstr_t("ManagedVst.PluginAdapter"), &pObjHandle);

Now we need to do some tedious unwrapping of all the COM layers around this object. Eventually we end up with a pointer to an interface matching that of the managed class (I defined this interface explicitly, but you can get the framework to create one for you).

VARIANT v;

VariantInit(&v);

pObjHandle->Unwrap(&v);

IPluginAdapter *pPluginAdapter;

v.pdispVal->QueryInterface(__uuidof(IPluginAdapter), (void**) &pPluginAdapter);

(Now you see why I want to write this plugin in C#?) So, finally, we have a pathway to a custom managed object which we can use and abuse at will. Next time, I’ll show how to marshal calls between the VST host and the managed plugin.

Eat Static

2004-10-11T17:13:00Z

I answered a question in a C# forum the other day from someone looking to do runtime subclassing, similar to NMock. The question went something like this:

"How do I overload the equals/not-equals operators for my dynamically-created class?"

Well, being hot shit at code generation, I immediately responded with something like this:

"It's easy - just define two static methods called 'op_Equality' and 'op_InEquality'".

Then I actually tried it. Although it's quite possible to define and emit these methods correctly, it doesn't do you much good. Operator overloads are resolved at compile-time, not at run-time, so for a statement like this:

bool areEqual = (foo1 == foo2);

if you have overloaded '==' and '!=' on type Foo in the usual manner (ie statically), the compiler will find the overload and emit MSIL similar to this:

ldloc.0

ldloc.1

call Foo::op_Equality(Foo, Foo)

If it doesn't find an overload, it will emit the regular comparison opcode:

ldloc.0

ldloc.1

ceq

And that's the problem. Because operators are static, there are no virtual calls to exploit - if your type doesn't define an overload at compile-time, you don't get a chance to "add" one at run-time and let late-binding find it. Same reason we don't mock static or non-virtual calls in NMock.

How big are your privates?

2004-09-15T13:05:00Z

There has recently been, and will continue to be, increased noise about Team System and its assorted components. The most widely discussed seems to be the unit testing piece, which is encouraging on one level because it means people are interested in unit testing. I have yet to see anyone ask the question "why?", however, as we already have several perfectly good testing frameworks (one of which VSTS blatently clones), with perfectly good (better than VSTS, even) integration with Visual Studio.

Lacking, also, is much contrarian opinion on the "Create Tests" feature in VSTS. I'll state my bias up-front - I think it's a Bad Thing - but I think if Microsoft are going to offer the feature, they should also provide guidance on how they think it should be used. I have several objections to this feature. First, it assumes that there is a one-to-one mapping between methods and tests (even down to the semi-configurable test-name generation). Second, it promotes a test-last culture. Third, it makes it super-easy to test private methods.

I'll expand on those points in reverse. Whether to test private methods is not one of those issues which has an easy, widely-accepted answer, but there are other significant questions which Microsoft should address before jumping off the fence:

* Is unit-testing about testing features or implementation?

* Should every piece of code within a class be testable via its public interface?

* Does coupling your tests to private implementation hinder or facilitate refactoring?

* Does the implementation of the xxxAccessor objects hinder or facilitate refactoring?

* Are private methods a code smell?

* Do framework developers have different API concerns to closed-audience developers?

I think providing an easy way to test private methods solves the wrong problem. Before reaching for the reflection, ask yourself why you can't test that method via the public methods which use it, and consider changing your class-under-test to be more easily testable.

The second point I consider the most important. Visual Studio (even 2005) already offers very little to the test driven developer, and I see "Create Tests" as a backward step. How about a "Create Code" feature instead? Whidbey offers "Generate Method Stub". Would it kill them to add Generate Class, Interface, Field, Local? And the expansions don't count - go and look at IntelliJ, or Eclipse, or even ReSharper. By making it so hard to drive development from your unit tests, it's no wonder that some developers view TDD, and programming by intention in general, with suspicion. Much easier to hack away inside a class and add tests later, that way the "flow" isn't interrupted with the constant flipping between unit tests, code-under-test, and the Solution Explorer. I was at a developer lab for VSTS recently and, I'm not kidding, the presenter offered Generate Method Stub as Visual Studio's support for TDD. He even dedicated a slide to it. I work with guys who would have laughed out loud at that.

Finally, the idea that each method on a class only needs one test method (one called 'SomeMethodTest' at that) should seem weird to any unit-tester, even those not doing TDD. What are you meant to do with the generated test methods? Put all your assertions about each method in one test? Add more tests as you go, thus instantly screwing up the naming conventions? What about tests which involve more than one method on the same class? What if you want to name your test methods after the behaviour they test, rather than the methods they exercise?

It just hasn't been thought through. If it had, they might have realised that providing lower level features to support code generation from the editor would have provided a far more flexible, and less dangerous, solution.

Visual Studio.Net 2001

2004-08-20T17:27:00Z

On the last project I was on, there were a lot of ex-Java guys who were used to working with IntelliJ and Eclipse, and who derived endless pleasure from ridiculing Visual Studio's comparative shortcomings. They renamed it 'Visual Retardio' ('Visual Subbuteo' came a close second). When Jetbrains started the ReSharper EAP earlier this year we all jumped on board in a flash, and stuck through months of pain and suffering (it was nicknamed 'ReBlunter' for a time) until finally the finished product was released.

Then Visual Studio 2005 went beta, and I was hoping that Microsoft would now have an IDE that was competetive with the best Java IDEs, and that would give us all the things we had become used to with ReSharper.

After using VS.Net 2005 for a few weeks, and having used VS.Net 2003 with ReSharper for the last six months or so, I have come to realise the following:

I much prefer VS.Net 2003 + ReSharper over VS.Net 2005, for the following reasons:

1) Refactoring

ReSharper's refactoring menu is context sensitive, and will only present those which are relevant to the current cursor location. 2005 just gives you all of them, then throws an error when you try and use one in the wrong place.
ReSharper has Introduce variable. I use this all the time.
ReSharper has a sweet inline rename for local variables and parameters, and will even suggest names for you based on the type.

2) Highlighting

ReSharper will colour fields, local variables and methods.
It will colour unused or unreachable code.
It will highlight all usages of a field/variable/method within a file.
Type this into any method in 2005:

Console.WroteLine();

What happens? Very little. Try it with ReSharper and it will colour 'WroteLine' in red, and present you with a drop-down list of all the valid Console methods (it will even narrow the list to the best matches).

4) Code Navigation

2005 now has a usable Goto Definition, but you can't navigate to sub or parent types.
2005 now has Find References, but it doesn't find references. It finds symbols matching the currently selected element, which is a very different thing. If I'm looking for references of a method, I don't want to be presented with a list of similar-looking methods on other classes, and I don't want the method I just selected to be present in that list. It's just broken.
Guess what? Both these features really work in ReSharper. And it has decent 'Find Type' and 'Find Method' feature, which narrows the list as you type.

5) Quick Fixes / Smart Tags

I'm no big-city usability expert <gasp!>, but to me, Smart Tags have a big failing: they require three steps to select an option via the keyboard. First, you use a hot-key to get the menu up. Then, you move the cursor down to the option you want, and then you hit return. This wouldn't be so bad, but if there's only one option in the list, you still have to navigate to it! ReSharper's Quick Fixes automatically select the first or only option in the list, and in some cases (if you use a type without importing it with a 'using' statement, for example) it will bring up a tooltip with the relevant option, requiring only one key press to activate it.

6) Intellisense

ReSharper will give you a list of types which haven't yet been imported. When you select one, it will complete your code and add the 'using' statement for you. It will also auto-complete variable assignments, including brackets and a semi-colon if required.

7) 'Miscellaneous'

ReSharper's 'Extend Selection' feature is quite hard to explain, but very useful. Essentially it's like ctrl-click, but it selects outwards from the current cursor position, and is scope-aware (eg it will select the name of a method call, then the call + its parameters, then anything enclosing that, then anything enclosing that, right up to the file level).
Duplicate Line - like [ctrl-c, cursor down, ctrl-v], but all in one go.

I really could go on, but this sounds too much like an advertisement already. Suffice to say that I am disappointed by the first beta of 2005, and I hope that Microsoft can improve things a little by the time they release. I am looking forward to the day when Visual Studio really understands code, rather than being a glorified text editor.

Double whammy

2004-08-18T17:01:00Z

Anyone who has passed a managed delegate as a parameter to an unmanaged function will be familiar with the oh-dear-the-GC-just-pulled-the-rug-from-under-me NullReferenceException you get if you don't keep save a reference to that delegate. It's hard to debug, but once you are aware of the problem you tend not to make the same mistake again. Or. So. I. Thought.

So I'm merrily hacking away with .Net 2.0, trying out the new flashy bits like generics and anonymous methods, when I tried out the new type-inference features in C#. Essentially this means that C# creates the delegate for you, and code like this:

button.Click += new ClickEventHandler(OnClick);

can be simplified like this:

button.Click += OnClick;

which is much nicer. It also means you can pass delegates to, you guessed it, unmanaged code without explicitly declaring a new delegate instance, and then spend a happy few hours debugging a stack-less NullReferenceException, safe in the knowledge that the problem can't be related to the delegate being garbage collected, because you'd never make that mistake again. In my defence, I was clearly drunk on syntactic sugar.

Which leads me to the second whammy: in 2.0, you don't always get a NullReferenceException in this situation. Sometimes, you get an AccessViolationException, which is a new type meant to help distinguish between null references and other types of AVs. Unfortunately, the CLR seems to be inconsistent about which one it chooses to throw (I was getting NRE on some runs and AVE on others) and led me to investigate the wrong flavour of bug.

Oh yes, we're fully XML enabled

2004-08-16T16:08:00Z

I missed one off the Programming by Superstition list:

Myth:

DataSets store their data internally as XML.

Rationale:

Microsoft said so!

When ADO.Net was announced, there was a lot of marketing guff about how it was "based around" XML, and how XML was a "first-class citizen". After all, it was the year 2000, and no CxO worth their salt was going to buy into a solution that wasn't "fully XML enabled". In fact, there is at least one article on MSDN which refers to DataSets being "natively an XML structure", so perhaps Microsoft believed their own hype.

Truth:

DataSets aren't really the key structure here - they don't do much more than group DataTables together. In turn, DataTables group DataColumns together, and DataRows act as a "horizontal slice" into a group of DataColumns. Going one level further, a DataColumn is just a wrapper around one of a number of classes inheriting from DataStorage (BooleanStorage, ByteStorage, CharStorage and so on), and those are just strongly-typed arrays with a bit of collection-management logic thrown on top of them. So, no XML. Sorry.

(And just for the record, they would have been feckin stupid to have used XML anyway).

Programming by superstition

2004-08-10T15:52:00Z

Myths and half-truths around .Net and C# best-practices crop up frequently in discussion forums, with rationales ranging from utterly believable to indefensible. I thought I would catalogue the ones I see most frequently. I realise I am leaving myself open to ridicule by prefixing my 'corrections' with 'Truth', but someone has to make a stand :-) Feel free to correct me.

Myth:

Always set your objects to null (or Nothing) when you're done with them.

Rationale:

The garbage collector won't be able to free an object if any another object is referencing it.

Truth:

The garbage collector can free any object that is unreachable from any of the various heap roots. If object A references object B, and object A is unreachable, object B can be released if it has no other references from reachable objects.

Myth:

StringBuilder.Append() is always faster than String.Concat().

Rationale:

String operations always create new string objects, whereas StringBuilder uses internal magic to avoid expensive allocations.

Truth:

Using a StringBuilder is sometimes (even usually) faster than using simple string concatenation, but there is a performance cutover point. Various people have found this to be around the 5 - 10 concatenations mark. The bottom line being that replacing this:

string fullName = firstName + " " + lastName;

with this:

StringBuilder builder = new StringBuilder();

builder.Append(firstName);

builder.Append(" ");

builder.Append(lastName);

string fullName = builder.ToString();

is rather pointless. Remember that StringBuilder is itself reference type, and has the associated allocation/deallocation costs.

Myth:

Strings are passed to, and returned from, methods by-value.

Rationale:

Strings are immutable, therefore they must have value type semantics, therefore must be copied by value.

Truth:

Strings have certain value-type semantics (namely, immutability), but are otherwise heap-based reference types, and references to them can indeed be passed or copied. Passing a string reference to or from a method does not copy the underlying string.

Myth:

Dispose() releases an object's memory.

Rationale:

Well, it's called 'Dispose' isn't it?

Truth:

Dispose() is merely a convenient place for a class' developer to place any resource clean-up code, in the hope that clients will call it. Dispose does not free the managed memory allocated for an object. The only thing that can do that is the garbage collector; there is no way to deterministically (or manually) release memory.

Myth:

Keeping ADO.Net connection objects open for the duration of a [Page/Control/1000-line method]'s lifetime is more efficient than repeatedly opening and closing it around each operation.

Rationale:

Connecting to a database is expensive, so try to avoid doing it too much.

Truth:

If your managed provider allows connection pooling, and if pooling is enabled (it is, by default, for the managed SQL Server provider at least), calling Close() on the connection object does not simply close the underlying connection. It merely releases that connection back to the pool where it can be associated with another managed connection object at some time in the future. This is a Good Thing, especially in multi-threaded environments such as ASP.Net, because physical connections are rare resources and should not be coveted. (By the same token, calling Open() on a managed connection when a pooled physical connection is available will simply associate the former with the latter).

By keeping a managed connection in an open state, you are preventing objects in other threads from getting hold of them, leading to potential bottlenecks.

Myth:

Int32 and int are not the same thing (ditto float/Short, long/Int64, string/String, etc.).

Rationale:

They're spelled differently.

Truth:

The C# simple type keywords are interchangeable with the equivalent BCL types; they are aliases.

The remarkable staying-power of of System.Assembly

2004-07-07T16:07:00Z

There was an interesting bug in NMock until recently (it's been fixed) which saw memory usage during unit test runs increase in relation to the total number of mock objects created during the run. The culprit was .Net's inability to unload assemblies from a running AppDomain: each mock was being hosted in a brand new dynamic assembly - which never gets garbage collected - so you could easily end up with a few hundred assemblies languishing in the root domain, eating memory. We fixed it by creating one assembly for all mock types, and by caching identical types for the duration of the test run.

Today, another bug was raised against NMock, which involved calling GetManifestResourceNames() on all loaded assemblies. Dynamic assembliles will throw NotSupportedException for this method (and for a couple of others), so if any code which relies on this call is used alongside NMock (in this case it was NVelocity), we break it.

I looked at a couple of workarounds, including saving our mock assembly to disk and reloading it as a static assembly so it could play nicely, but noticed that AssemblyBuilder, which we use to generate the mock assembly, is never garbage collected either! So, kids, be aware that if you generate assemblies at runtime, even just to save them to disk, your AssemblyBuilders will hang around for ever, holding you memory hostage and calling you "poo-breath" behind your back. The solution, of course, is to do your code-gen in a separate AppDomain, which is pretty weak but at least you can throw it away when you're done.

Does anyone have a good reason why assemblies cannot be garbage collected?