It’s been a while and I said before I wouldn’t post anything again regarding data-access benchmarks, but people have convinced me to continue with this as it has value and ignore the haters. So! Here we are. I expect you to read / know the disclaimer and understand what this benchmark is solely about (and thus also what it’s not about) in the post above.
The RawDataAccessBencher code has been updated a couple of times since I posted the last time, and it’s more refined now, with better reporting, more ORMs and more features, like eager loading.
The latest results can be found here. A couple of things of note, in random order:
- Entity Framework 7 RC1 (which we used here), is slow, but later builds are faster. It’s still not going to top any chart, but it’s currently faster than EF6, according to tests with a local build. We’ll update the benchmark with results from RC2 when it’s released.
- LLBLGen Pro v5.0, which is close to beta, has made a step forward with respect to performance, compared to the current version, v4.2. I’ve optimized in particular the non-change tracking projections as there was some room for improvement without cutting corners with respect to features. The results shown are achieved without generating any IL manually. The performance is better than I’d ever hoped to achieve, so I’m very pleased with the result.
- The NHibernate eager load results are likely sub-optimal, looking at the queries, however I couldn’t find a way to define a more optimal query in their (non-existing) docs. If someone has a way to create a more optimal query, please post a PR on GitHub
- The DNX build of the benchmark currently doesn’t seem to work, at least I can’t get it to start. This is likely due to the fact it was written for Beta8 and current bits are on RC1 and tooling changed a lot. As their tooling will change again before RTM, I’ll leave it at this for now and will look at it when DNX RTMs.
- The eager loading uses a 3-node graph: SalesOrderHeader (parent) and two related elements: Customer (m:1, so each SalesOrderHeader has one related Customer) and SalesOrderDetail (1:n). The graph is a graph with 2 edges which means frameworks using joins will run in a bit of a disadvantage, as the shortcoming of that approach is brought to light. The eager load benchmark fetches 1000 parents.
- The eager loading only benches change tracking fetches and only on full ORMs. I am aware that e.g. Dapper has a feature to materialize related elements using a joined set, however it would require pre-defining the query on the related elements, which is actually a job the ORM should do, hence I decided not to do this for now. Perhaps in the future.
- The new speed king seems to be Linq to DB, it’s very close to the hand-written materializer, which is a big achievement. I have no idea how it stacks up against the other micros in terms of features however.
I almost forgot to show an interesting graph, which is taken with dotMemory profiler from Jetbrains during a separate run of the benchmarks (so not the one taking the results as profiling slows things down). It clearly shows Entity Framework 7 RC1 has a serious memory leak:
As some people can’t view pastebin links, I’ve included all results (also from the past) as local files to the github repository.