.NET O/RM Performance Comparison

With the launch of the ORMBattle.NET site, a new discussion started on the blogosphere; later on, this post by Gergely Orosz added more ashes to the fire. The subject seems to be, is it possible to blindly compare O/RM tools, disregarding all differences between them, in simple yet not plausible scenarios, such as loading/saving/updating N entities in a loop?

Oren Eini wrote about it, and so did Davy Brion, both dismissing the way tests were conducted and the general usefulness of these kinds of (well, useless) tests. I tend to agree with both, but I don't know why the performance of these OR/M tools (which all implement the Unit of Work pattern), properly configured of course, can't be configured in simple but common scenarios, such as loading entities by id, loading an amount of entities at the same time, entity materialization, entity saving and updating, SQL generation quality, and so on. Of course, a more interesting test would be implementing a simple application with well known requirements with each technology, something like the PetShop application. Or, why don't we just forget about LINQ (NHibernate's implementation, at least, is not very mature), which seems to me just a trendy subject, and focus instead on things like:

• Cache support
• Available mapping types
• Support for multiple tiers
• Query language quality (or existence)
• Possibility to pre-compile queries
• Support for stored procedures
• Memory and CPU consumption
• Existence and quality of supporting tools (automatic entity generation, for example)
• ...