O/R Mappers: Avoiding Reflection

Steve Eichert posted his findings yesterday about the performance cost of reflection.  I knew reflection was slower, but I had no clue it was that bad potentially.  I haven't done many tests yet myself to see if it really is that bad, but it doesn't really matter since I can agree that reflection is definitely slow.  So why does this matter -- well, right now my WilsonORMapper uses a lot of reflection to get and set the field values.  I was planning on doing something to fix that sooner or later, but Steve's post got me thinking about making it my next priority.

So how should I go about getting rid of reflection?  The first solution I came up with is the easiest for me to implement, although I'll admit it seems kind of kludgy and ugly to the user.  Basically, I would provide an interface with one property whose single parameter would be the member name specified in the mapping file.  The user (or my WilsonORHelper) would simply implement this interface's property with a switch block where they set or get their own member variable to avoid reflection if performance is a consideration.  I would simply need to use reflection one time, on the initial load of the mappings, to see if they implemented this interface.

OK, that does sound pretty kludgy, and it does mean that I would be requiring the user to do something specific for a change.  But this would not be "required" unless they want or need the performance gain, and its still not requiring them to inherit from a specific class either.  Implementing an interface, while definitely a requirement, is not as big of a "burden" since you can have multiple interface inheritance in .NET.  And again, its not really required unless the user really wants or needs the performance, which may be necessary for collections of many objects, but maybe not for other cases.

So what other options exist to avoid reflection?  One option that Steve mentioned is to use CodeDOM to dynamically create an assembly with a signature that the O/R mapping framework understands that knows how to call the public members or properties of the original class.  Those public members or properties might be specified totally in the mappings, or reflection might need to used one time at startup at most.  The problem I have with this technique is that it requires public members or properties, and it doesn't handle any member that is read-only publicly.  Assuming that there aren't many read-only cases, what's the problem with using public properties, since they almost always exist anyhow.  The problem is that properties are often (and should be) wrappers around the private fields that contain additional validation or business logic.  There's nothing wrong with your public property rejecting or modifying the user's attempt to set an invalid value, but it should not prevent me from loading data that currently exists in the database.

So what do other O/R mappers do instead?  Some O/R mappers use CodeDOM to dynamically create an assembly with new classes that inherit from the original classes which become the real ones used by the mapping framework.  This can be done by either having the original classes be abstract with all the real logic created in the new dynamic inherited class, or by requiring the original class to expose its member variables as protected.  The problems with this approach are that you can't use new to explicitly create your classes anymore, and the classes that the framework returns are actually a different type technically than was originally expected.  Neither of those are significant problems, but the requirements to do this aren't trivial.  You either have to forego providing your own implementation that includes your validation and business logic with an abstract class, or you have to expose all of your member variables as protected, and neither of those requirements are very friendly.  There may be solutions to this that some O/R mapping frameworks have discovered, so I'm not trying to imply there isn't, but I doubt any such solutions are trivial, and they are probably therefore out of my reach to easily implement.

What other solutions are used by O/R mappers?  Some O/R mappers require you to generate lots of code in order for them to work so that they don't have to use reflection and so that they can also gain other "insider" knowledge.  I don't want to imply that this is "bad", or not a valid technique for O/R mappers, both because that's not necessarily the case, and because there have been other discussions on this already.  That said, its not what I want to do with my O/R mapper, so this was not ever an alternative I seriously considered.  One thing it does do for me, however, is to at least validate that allowing the user to implement an optional interface with a single property if they want or need the extra performance is not something totally out of line with other tools.  And since I can make my WilsonORHelper generate this code if the user wants to use my helper and wants to turn on this feature, then I do feel like its not at all too much of a "burden".

So at this point I've just about concluded that my first solution is probably good enough, at least for my simple O/R mapper, and I also have decided that I don't really like the other alternatives, at least that I can think of or find.  Then it occurred to me that I should try to figure out what ObjectSpaces is doing, but I quickly gave up since their code is just too much for me to try to figure out without lots of time and work.  Then, on a whim, I decided to  Google on ObjectSpaces and reflection.  The first result was a blog entry by Andres Aguiar that confirms ObjectSpaces does use lots of reflection, but this is somehow going to be less of a performance hit in .NET v2.0.  The fifth result returns some documentation about ObjectSpaces and an IObjectHelper interface that I had never noticed before -- and remarkably it sounds exactly like what I was proposing to do!  There's also an IObjectNotification interface that can be implemented to enable your objects to receive events when is updated or deleted or when an exception occurs, which was something else I was wanting to do somehow.

That's enough research for me, since I liked my solution to begin with, and since I'm mostly using the syntax of ObjectSpaces anyhow, this will now definitely be the thing I implement in the next few week or so.  Of course, that still doesn't mean its the best or "right" solution, so I'm still interested in what others think of my solution and the other options.

Published Sunday, January 11, 2004 8:58 PM by PaulWilson

Comments

# re: O/R Mappers: Avoiding Reflection

Before you start to worry about redesigning the mapping layer, I would suggest you sit down and actually stress test your code.

If it is orders of magnitude slower, perhaps a redesign/rethink is in order. However, a mapping layer, by nature, is going to add performance overheads to your system. That is the tradeoff with the mapper. Ease of use versus speed of execution.

A reflection based mapping layer is never going to come close to hand rolled sql + hand written object instantiation code.

Yes reflection is slow. No, it really isn't as slow as everyone thinks it is. (Take a look at the asp.net runtime code... theres a lot of reflection going on there)

Sunday, January 11, 2004 10:00 PM by Sean Malloy

# re: O/R Mappers: Avoiding Reflection

I designed my O/R mapping architecture so that it's essentially pluggable. The default mapper uses Reflection. Not only does it use reflection, but right now it doesn't even try to be performance friendly and cache the FieldInfo, it pulls it every time. That's something I'll fix when I start to worry about performance. The other thing I was gonna do is still use reflection, but instead of looping over the fields, generate a mapping helper class per persisted entity type with the CodeDOM. A third approach certainly could be the same thing as the IObjectHelper idea. The point is though, it's completely pluggable. You can choose what mapping technology is used for your application via the configuration file.

Sunday, January 11, 2004 10:19 PM by Drew Marsh

# re: O/R Mappers: Avoiding Reflection

Paul, while on the surface I can agree that, - to be pedantic - it might seem like a good idea to increase the performance of your OR mapper by replacing the Reflection, don't you kind of think that this is exactly the type of problem that Reflection was built to handle? It's probably not as if you are going to be handling MILLIONS of calls per minute :-) and having the Reflection cause bottlenecks.

Either way, I'll bet that the bottleneck caused by the Reflective calls are significantly less than the bottlenecks caused by the system you are replacing :-) - I.e.: hand coding.

Sunday, January 11, 2004 10:21 PM by Darren Neimke

# re: O/R Mappers: Avoiding Reflection

Implement the Data Transfer pattern. This will force you to create a 'fields' object which is in fact the object which contains the data which is persisted and which is filled by your data-access code. This fields object is 'plugged' into an entity object and the entity object then is populated with its own data. The fields object contains 'entityfield' objects which are addressable by index (int) or name (string), and contain the actual data of the field, persistence info (if you want, you can store that separately, like you have in an XML file) and some code to deal with IEditableObject (very important).

This solves your problem. CustomerEntity.CustomerID is a property which indexes into the _fields object of CustomerEntity, passing "CustomerID" or CustomerEntityFields.CustomerID as index and sets/gets _fields[index].Value.

No reflection needed, everything is taken care of by base class code like IEditableObject, rollbacks after a transaction (also very important: two autonumber entities are updated in a transaction, the first succeeds, gets the key value, second fails, the first has to roll back in memory to the old values!!).

Here's a good article which describes this feature (in a somewhat different structure, it's a java O/R mapper)
http://www.uq.net.au/~zzabergl/simpleorm/whitepaper.html

Sun's pattern pages about Data Transfer pattern:
http://java.sun.com/blueprints/corej2eepatterns/Patterns/TransferObject.html
Sun's pattern pages for J2EE:
http://java.sun.com/blueprints/corej2eepatterns/Patterns/index.html

No MS pattern pages? Well, look at the Sun pages and you know why.

Monday, January 12, 2004 3:51 AM by Frans Bouma

# re: O/R Mappers: Avoiding Reflection

Frans,

My old O/R mapping layer (non-.NET) was implemented using that pattern, but I felt like it didn't do enough for the user. I believe the user just wants to work with their own strongly typed fields. I believe that asking the user to write a ton of casting logic in their entity class' properties/methods is a pain in the butt for users. I believe the complexity should be in the O/R mapper, not in the entity objects themselves. As for the example of versioning the fields, that's easy enough to do within the mapping layer itself. Obviously the trade off is performance. There's a billion ways to skin the O/R Mapping cat. I guess it will be up to the users to choose a framework that makes the most sense to them.

Monday, January 12, 2004 12:38 PM by Drew Marsh

# re: O/R Mappers: Avoiding Reflection

What's your goal with this O/R Mapper - to figure out how things work, or are would you like to use it in a production environment?

Tuesday, January 13, 2004 1:00 PM by Jerry Dennany

# re: O/R Mappers: Avoiding Reflection

Hey Jerry:

While I am interested greatly in how things work, I am also interested in more than that. First, my site (http://www.WilsonDotNet.com) is largely an example of best practices, and O/R mapping is something that I wanted to include because I think it is often (but not always) a best practice. I am working on performance now, and let's just say that the numbers are really very good (which I'll be blogging and announcing in the next few days), so I also do plan on using this O/R mapper in "real" production systems beyond my site. That's not to say that it will suffice for all cases or everyone that needs an O/R mapper, since I won't be including all possible features.

Thanks, Paul Wilson

Tuesday, January 13, 2004 1:59 PM by Paul Wilson

# re: O/R Mappers: Avoiding Reflection

I would recommend taking a look at the architecture of two Java-based solutions, Gemstone Facets and JDO.

Facets has persistent hooks built right into the JVM; Microsoft has the capability of doing that, as there is only one CLR, and it is the ULTIMATE as far as transparent persistence goes in the JVM or CLR world. It is not, however, practical for just anyone to do, as you'd have to get access to the CLR source code.

JDO originally attempted to take the Facets direction, but the spec group got shut down by the JVM spec group, saying that there were too many JVM implementations out there to change if persistence hooks were to be included in the JVM spec. So, the JDO architecture does the next best thing: if the JVM won't supply the extra bytecodes for field interception et al, then a tool-supplied enhancer will.

It would work like this in a .NET environment. You supply the enhancer with the assemblies that contain your persistent classes, along with a persistence descriptor file, and the enhancer spits out new, modified assemblies that contain persistence capable classes. Debug information is preserved, and it's a build-time event, not a run-time event.

The upshot?
* No use of reflection at run-time,
* no requirement for public things that really shouldn't be public,
* built-in lazy loading without having to use IObjectHolder to do the lazy loading, and
* the ability to decaratively use spans (and other things like transaction isolation levels, etc).

There is already an upcoming product from Poet called FastObjects.NET that uses the enhancement strategy successfully. See http://community.fastobjects.com/community_fastobjects_net.htm for more information. I would love to see Microsoft take the persistent CLR direction, but, if not, at least go the enhancement route. The impact on the developer is that persistence is very much transparent, leaving the drudgery to the tool.

--matthew

Tuesday, January 13, 2004 6:44 PM by Matthew Adams

# O/R Mappers: Maximum Performance

Wednesday, January 14, 2004 3:53 PM by TrackBack

# re: O/R Mappers: Avoiding Reflection

All right, I've got my performance up and comparable to DataSets now, beating them in some cases.

Later, Paul Wilson

Wednesday, January 14, 2004 4:21 PM by Paul Wilson

# re: O/R Mappers: Avoiding Reflection

Why not to save property info object for any BusinessEntity in shared hashtable and retrieve it from there when ever we need to set the value for any property through reflection.

Tuesday, May 04, 2004 3:49 PM by Murad Kayani

# re: O/R Mappers: Avoiding Reflection

As Steve noted, and I verified, most of the performance hit occurs when actually "using" reflection, regardless of whether or not the reflection "object" was reused or not. I actually do reuse the reflection "objects" to get the small amount of efficiency this provides, but its simply not at all the same as direct access, so I must recommend the interface approach when performance is really needed.

Tuesday, May 04, 2004 4:24 PM by Paul Wilson

# re: O/R Mappers: Avoiding Reflection

Yes Paul, you are right, I tested that and there is a small performance gain you get, but during the test, I found one interesting thing that if you set private member through reflection it takes a lot more then setting public member, any idea why it is so?

Wednesday, May 05, 2004 4:15 PM by Murad Kayani

# re: O/R Mappers: Avoiding Reflection

Yes that was also noted in earlier results by Steve -- I believe security was suggested as the reason since you have additional security checks to see if you can reflect on private members.

Wednesday, May 05, 2004 5:00 PM by Paul Wilson

# J2EE patterns - a nice read too

Monday, August 30, 2004 7:24 AM by TrackBack

Leave a Comment

(required) 
(required) 
(optional)
(required)