Brian Goetz on micro-benchmarks
Brian Goetz writes on micro-benchmarks
and discourages people from writing any. The first part of
the article details why some particular banchmark is flawed,
which is only mildly interesting unless you're interested in
lock performance in Java (and why wouldn't you be?), but the
second part gives excellent advice on performance testing in
general.
I thought I would give a pointer to his article as I've been
guilty of micro-benchmarking myself on this blog more than
once. To my defense, I've always said that these gave only a
rough idea of performance in a real-life scenario, and that
any performance testing should be made in the context of the
real application.
The problem for us API developers is that our users ask for
guidance on when to use this or that particular technique.
We can still give some general answers based on reasonable
micro-benchmarking and analysis of the IL code in some very
simple cases, and that's enough for most users. But nothing
will ever replace a good profiler and a lot of
experimentation on the real application when it's being used
in real-life conditions.
Read the article here: