January 2004 - Posts

Looking for ASP.NET Article Ideas

ASP.NET Pro Magazine is looking for articles.  Does anyone have any ideas for articles you would like to see written?  Or if you prefer, you can try to propose one yourself to the editors.
Posted by PaulWilson | 9 comment(s)

Mixing Forms and Windows Security in ASP.NET

My latest article, Mixing Forms and Windows Security in ASP.NET, has finally been published on MSDN's ASP.NET Developer Center -- yeah!  I see questions almost daily in the forums from people asking how to combine Forms and Windows authentication -- so here's the answer finally.  This article details how to use Windows authentication for Intranet users, including capturing their username, while still providing a custom login screen for Internet users.  Thanks Kent for making this happen.

Posted by PaulWilson | 3 comment(s)

WilsonORMapper, EntityBroker, and LLBLGenPro

I've apparently really annoyed Frans, and Thomas a little too, so I thought I would do something different. First, let me just say that I've never spoken bad about either EntityBroker or LLBLGenPro that I can recall. In fact, I said publicly many times that both of their products are very good and highly recommended. Why? Because as Frans noted, my WilsonORMapper is "severely crippled" if you need more than simple CRUD. That's right -- my O/R mapper is a simple tool for simple minds only -- and that's all I want it to be. Yes, I want to add cool features, and yes I have a blog to announce it, but I try to also share what I learned in the process. Does that make me arrogant? I hope not, but its not the first time I've been called that, so I'm sorry and I'll try harder, but I hope my accusers can also try a little harder please.

So, before telling you more about EntityBroker and LLBLGenPro, I'd like to share with Frans what I have done right. First, I have done a great job of getting the O/R mapper message out there, as have many others like Steve. If that weren't the case then I wouldn't be getting so much positive email in the last few weeks. In fact, several have went out of there way to tell me how much the hostile attitudes of Frans and Thomas had turned them off! Yes, I have found that there are many people out there that just want a simple O/R mapper, which is where mine excels, but I have also recommended others to EntityBroker and LLBLGenPro while admitting mine wasn't up to their needs. So I think I'm helping the cause in general, and I'm probably even going to increase business for Frans and Thomas to some degree.

Finally, if my WilsonORMapper is so simple, what is it that EntityBroker and LLBLGenPro do that I don't? Both include "real" GUI designers, as will MS ObjectSpaces, whereas my ORHelper merely helps. Both support COM+ transactions, multiple concurrency types (supported now), full data-binding, object query APIs (some support now), cascading persistence (supported now), dirty-field only updates (supported now), and much better support and documentation. EntityBroker supports the major databases already, and I think I just got Thomas to start thinking MySQL seriously too now. LLBLGenPro has plans in the works to support Access, MySQL, and more, so its always been only MS ObjectSpaces (and some others) that I've criticized vocally in that regard. EntityBroker does some really advanced caching, and I have personally used it enough to say I really like it, although it is too "complex" for many simple cases.

I haven't ever used LLBLGenPro personally, since its not really my style, but it looks very good so I do recommend it too. Its designer will automatically discover your relationships, which seems extremely helpful. It also supports stored procedures, as does mine, so again I've never intended to criticize it in this regard. LLBLGenPro also supports things like joins, group bys, validation, and many advanced Oracle features I can only dream about. It also already supports multiple field primary keys, which is something I still need to try to get to. Do I think Frans should use only generic ANSI SQL? No, I've never said that -- I've simply said that starting with generic ANSI SQL will automatically get you working with many databases in the simple cases.

And that's what I don't get -- why are so many O/R vendors totally ignoring the simple users?  Are we really going to convince people they need an O/R mapper by raising the bar so high to just get started?  I still can't personally understand half of what Frans and Thomas list as their features, and they focus so much on these that the simple ones aren't obvious at all.  Many of the other vendors have much better explanations of the benefits of O/R mapping, but most of them do a terrible job at more than code generation.  I want people to really understand how O/R mapping can help them, and if they go buy someone else's then that's fine too.  But I don't really see how all the antagonistic debates in forums and blog comments make any of these points clear to most users.

WilsonORMapper works well with MySQL

I spent a couple of hours yesterday testing my WilsonORMapper with MySQL (note the My, not MS). I designed my O/R mapper to work with any ANSI compliant database, so I had high hopes. Note that I do of course have a few optimizations for MS SQL, Access, and Oracle, so I don't mean everything is or should be generic. I'm very experienced with Oracle, although its been a few years back, but I had never touched MySQL until now. I started by downloading MySQL v4.0.17, which was the recommended version, MySQL Control Center, and MySQL Odbc Driver v3.51. I installed everything, then I figured out how to start the server and create a database and a table pretty easily. The next thing was configuring Odbc and figuring out my connection string, and do a few small regular ADO.NET tests. So far so good -- by the way, MySQL seems surprisingly comparable to MS SQL in ease of use.

So next I tried my first test with my WilsonORMapper, a simple GetObject, and it failed. My debug version spits out all the SQL so I copied it into the query tool to test it directly since it looked good. It turns out that MySQL is not ANSI compliant by default -- it uses ` instead of " for its table and field delimiters. By the way, neither is MS SQL, which uses [ and ] by default, so my Commands class allows this to be easily overridden. I tried to get the Option clause in my connection string to turn on ANSI support, but finally gave up and just started MySQL in ANSI mode (mysqld --ansi). That was easy enough so now all my GetXXX methods work great, just as expected when you support generic ANSI. Note that I will still need to write a custom Commands provider to get paging to work, but that will be a simple method override.

Next, I tried to do an Insert and it failed at first, but this time the error message in the debugger helped right away. I use parameterized dynamic sql for all my inserts, updates, and deletes, which stops sql injection attacks and allows the database to cache the execution plan. The Access OleDb driver supports named parameters, not just ?, although you still have to assign your values in the same order as the parameters, so that's what I have coded. Well the MySQL Odbc driver requires ? for parameters -- but guess what, I have an optional parameter mapping for all my fields since I support stored procedures. A simple change to the mapping xml file and now all my inserts, updates, and deletes work with MySQL too. Note that auto-generated identities are not supported by ANSI SQL, so I have a hack using MAX in place that isn't scalable, but a custom Commands provider can easily fix this with again a simple method override.

So I would like to say that my WilsonORMapper supports MySQL, but I really should probably write that custom provider to make it perfect. That brings me to a few questions though, which maybe some of my readers can help me better answer. There are multiple MySQL .NET native drivers out there, as well as OleDb and Odbc drivers -- what should I expect of my users? Should I force a specific driver on them or should I just write a custom Commands provider and expect them to do something more to take advantage of it? And if the latter, should they have to recompile my source with their few driver specifics, or should I somehow make it a configuration option? The same questions will apply to PostgreSQL also, which I have someone asking me about already, so whatever I decide will apply to other databases also.

By the way, to me this experiment validates a couple of my design decisions that I blogged about previously. First, generic ANSI SQL is not a silly pipe dream -- it really is possible in most simple cases, although certainly you should provide provider specific optimizations too. Yes, I know I'm just talking about the simple CRUD operations, but apparently I've guessed correctly that this really is all many people care about, judging by the responses I have received. Finally, having my mappings in an external xml file was crucial here, since I needed to change my parameter names to ? for just MySQL, not all the other cases. This means that I did not have to recompile my code -- all I needed was a MySQL specific mapping file and everything worked perfectly. If O/R mappers are to support multiple database platforms, then how can we honestly ever expect the mappings to only ever be specified once in the code as attributes?

Posted by PaulWilson | 4 comment(s)

WilsonORMapper now tested with Oracle

I just finished updating my WilsonORMapper to v1.1 today.  I added support for many-to-one and many-to-many relationships (already had the one-to-many case) which proved pretty painless.  I also added support for some common requests, like loading object collections with stored procedures (already supported stored procs for insert, updates, and deletes), loading datasets with only a subset of fields (instead of always loading all fields), and directly executing set-based updates and deletes (along with expressions for updates).  Anyhow, the hardest thing was that I finally got around to testing everything with Oracle 9i, so everything now works with Sql Server, Access, and Oracle (including my paged collections).  I'm using a new feature of Oracle 9i to do my inserts that rely on database-generated keys, the RETURN INTO clause, which assumes that a trigger exists that gets the next value from the appropriate sequence and includes it in the insert -- almost like identities in Sql Server and Access.  I also did go ahead and break out my “Commands” class into a provider model that makes it easy to replace my ANSI SQL for the special cases in Sql Server, Access, and Oracle, while still supporting the generic cases for all other databases!
Posted by PaulWilson | 2 comment(s)

Running .NET WinForm Applications on Citrix

We have been creating a major .NET WinForm application that will be deployed in Citrix for a few months now.  Basically, think of it as a custom rewrite of MS Access (note that this wasn't a techie's idea) that works against MS SQL Server.  Of course, business as usual doesn't like any limitations, so we have way too much data and far too many features to work in a shared environment like Citrix.  Or do we?  That has been the bane of my existence for the last couple of months, and unfortunately there doesn't appear to be many others that have such experience to share.  My main concern has been whether or not the .NET garbage collector would see the 4GB of RAM and think it could have it all, unaware of the other users on the box.  I have posted that concern to various forums several times, and I've seen a few others with similar concerns, but no one ever answered, other than people telling about .NET works in general without any Citrix experience.

Well I can finally report that things look pretty good afterall -- after much painful performance tuning and tracking down a memory leak in a 3rd party grid control.  There were many small performance gains, but the biggest one is something I still don't understand, but the numbers are proof.  Basically, we have a business class that has a method that gets a DataSet and stores in one of its members, and then the calling class separately sets the grid's datasource to the class property that exposes that member.  It seems like that would be fine, since a DataSet is a reference type, but it turns out that setting both the class member and the grid datasource in separate steps takes twice as much time as having the class method set the member and then return it to the caller!  I would love it if someone can explain that one to me -- my team told me it didn't work this way, but the numbers are proof, and none of us understand it.

So we were hoping for good results with our test this week -- sure enough, the first 45 minutes testing the application in Citrix with 15 users looked very good from a user's point of view.  But then things started grinding to a halt, and didn't pick up for 30 minutes when most of us started closing and restarting the application.  The naysayers in our company quickly released their observations that our application could not even support 15 users, which isn't very good in an expensive Citrix environment.  The infrastructure guys pointed to some poor .NET performance counters -- but wait, those numbers didn't get bad until 15 minutes after everything went to crap, and all the .NET numbers before then looked sweet.  The problem instead is obvious when you look at the overall memory numbers -- there was a gradual memory leak, unrelated to .NET, that eventually swamped the system, and eventually that also affected the .NET numbers.  I finally found the control, the 3rd party grid, that had the memory leak, and it turns out they already knew about this and had a hotfix, not to mention a newer version that we were still investigating.

We will now start updating that grid control, and I hope to soon be able to report a better test result in Citrix.  There may still be problems, but the numbers once again speak for themselves, and I think its pretty clear that our .NET app ran better than Access and our older Access-like applications that we are trying to replace.  The naysayers see .NET start with 20MB and freak out saying that its not going to scale, but that initial grab doesn't mean it takes 20MB to run the app, it just means .NET is getting memory so it doesn't have to later.  Its actually quite an incredible feeling to look at the graphs of .NET's memory and garbage collection when you have 15 people on a Citrix box, each looking at about 3 separate instances of our main view, each having an average of 25000 rows of data (some much more), averaging 25 columns (but some with more again).  .NET works great, and so does MS SQL Server, which we use exclusively with some of the world's largest databases -- that's right, we don't have any instances of Oracle at all.  Life is good.

Update: Life is NOT good since none of the global .NET memory performance counters that I relied on actually work!

Moving from Wise to VS.NET Deployment

I just finished ditching my company's use of Wise Installer for .NET. I replaced it with the built-in Setup and Deployment project of Visual Studio .NET. Why did we choose Wise to begin with? Mostly ignorance -- I'm not a build engineer, and I was told to pick Wise or InstallShield. I was sadly too ignorant to even consider the VS.NET built-in tool. I had read it was better than the one in VB6, but the VB6 one was so bad that I just assumed VS.NET was still not up to par. I assume the person that told me to pick Wise or InstallShield has similar excuses, but he's gone now. Anyhow, my "upgrade" to the free VS.NET installer from the much more expensive Wise installer is now done -- and I'm glad to be rid of Wise.

What was wrong with Wise? First, I need to state that its probably just as good, if not better, if you are a build engineer that knows WiseScript, and the idiosyncrasies of Wise. I assume something similar can be said about InstallShield, but I don't have any recent experience with it to even talk about. The first problem I had with Wise was that it kept removing valid references or adding unnecessary ones, including duplicates, which became very annoying. I also started to experience crashes when I was doing several things, which forced me to start saving after every change. Then there was the fact that I simply don't know WiseScript, so I had no idea how to do the custom things that were starting to arise. The final blow was that Wise, at least not in my simpleton attempts, did not install assemblies in the GAC when it was told to do so! All the help documentation, and rave reviews, say its trivial -- just drag the file into the GAC folder in Wise -- but that did not work for us. My one attempt to fix this in a workaround by including GacUtil in the install, and then calling it to manually register my assemblies with one of their custom actions, did not work either. Again, maybe it was all a case of my being a stupid user, and I'll admit that I also never called Wise for help.

So why the VS.NET Installer? First, I started hearing more people ask me why I wasn't using the one built into VS.NET, which got me at least thinking. Then I read an article somewhere that really opened my eyes -- sorry, I can't find it anywhere now. So I finally decided to convert my Wise project to a VS.NET Deployment project, which the wizard in VS.NET made very easy to do. I now have my assemblies getting deployed to the GAC as effortlessly as should be, and I haven't encountered many idiosyncrasies, at least none that keep happening. One problem I found is that the deployment packages won't automatically uninstall old versions, even when you tell them to check for them they just stop and tell you to do the uninstall yourself. The fix for this was to change both the version number and the GUID product code of the deployment project itself before every build. This was easy to automate since the project file is plain text, unlike Wise's project file, so I just have a command-line app that is called from my batch file that changes these things, much like it already changed the application version numbers in all my AssemblyInfo files. I also no longer have a single EXE that includes the .NET framework, but that's not a big deal with the Setup.exe bootstrapper, so I can live with that. The main thing is that now I look forward to working with it, since I can use C# or VB to write my custom actions in the future, and therefore be far more productive. Its also nice that its free (if you have VS.NET), so all of our developers can open the deployment project now if necessary, and all of our other projects can also use it, so I guess I can even say I've saved my company thousands of dollars!

By the way, my batch file I referred to starts each build by deleting all my old code and binaries, then it gets everything from Visual SourceSafe, either from a label or the latest depending on settings.  Then it changes all the version numbers, before building all binaries, finishing with the build of the deployment project.  Finally it copies the installation files to an appropriate location.

Posted by PaulWilson | 7 comment(s)

SOA Is Not New -- Its Just Easier and "In"

Steve and others have been talking about SOA and it got me thinking that we basically had an SOA architecture where I previously worked (Roche Diagnostics), we just didn't know that was the terminology then.  We had a GUI application that needed to talk to a variable number of medical devices, which typically had a life of their own, so that any given task initiated from the GUI could take anywhere from a second or two to 15 minutes.  They could also start doing things on their own in some cases, and we also needed to be able to easily integrate new devices on the fly as customers bought more devices, or when we came out with new ones.  The first thing we did to solve this was specify that each device could only talk to its own "controller", which knew its own unique features (or problems), that ran in its own process.  Then we defined a set of common interfaces that the GUI and each controller had to do all of their communication through, which we then registered with the COM+ Event subsystem.  This allowed us to add new devices on the fly very easily, or even "test or monitoring controllers, since neither the GUI nor any of the controllers ever talked directly to each other.  We also specified that all communication with the COM+ Events interface was done through MSMQ Queued Components, which allowed everything to occur in its own time asynchronously, without blocking, while still being guaranteed.  Finally, we also had an "aspect" service for event logging and tracing, which was totally configurable to allow us to turn on different levels of logging, for all or just some devices.  This may not have been anything "sexy" by today's standards (no .NET or web services), but it was very much a service-oriented architecture, built on COM and COM+, taking advantage of COM+ Events and MSMQ.  The only thing that I never really liked about our architecture was our data tier, which always felt way too convoluted with its stored procedures that were wrapped by COM+ components, that were finally called by the GUI.  It was too late when I started learning about O/R mappers, which would have greatly simplified things by basically making an entity service.  Of course I would have had a hard time selling the O/R mapping stuff at Roche, but then again maybe not since the business upside would have been that we could have sold our product with any database rather easily.  So in my opinion SOA systems are not anything new, although there are certainly new ways to build them, which are easier than ever, although the downside of this is that many people are building SOA systems that don't need them.  Anyhow, the principles have been around for a long time -- loosely coupled contracts, whether that be interfaces or web services, often with asynchronous independence, possibly with guaranteed delivery, running in different processes or machines, or possibly even different networks.
Posted by PaulWilson | with no comments

O/R Mappers: Maximum Performance

I now have my WilsonORMapper (v1.0.0.3) performance very comparable to that of DataSets. In some cases I am actually beating DataSets, with or without private field reflection.

My tests compared DataReaders, DataSets, and my ObjectSets, both using my new optional IObjectHelper interface for direct access, as well as testing private field reflection. Each run consisted of a number of repetitions of loading the appropriate structure from a database table and then looping through the resulting structure to access the values. The database table consisted of 10,000 records filled with random data that I generated, with the table fields consisting of 2 ints, 3 strings, 1 float, 1 datetime, and 1 bit. The numbers posted all represent loading 10,000 records, but cases varied from 1 record repeated 10,000 times, to 100 records 100 times, and finally 10,000 records only 1 time. The tests were ran many different times, and the numbers were always consistently similar. I also tested a 100,000 record table, and the numbers were similar, just 10 times bigger.

Notice first that hitting the database many times for one record is noticeable slower. Next, note that DataSets are pretty much always twice as slow as using a DataRepeater. If you want to load a single record then my WilsonORMapper beats a DataSet hands down. This remains true even in the case where I continued to use private field reflection. On the other hand, my O/R mapping was 50% slower than the DataSet loading 100 records, and 75% slower than the DataSet when 10,000 records were loaded, using direct access. The numbers were another 2 times slower when I allowed the private field reflection. So performance varies depending on the number of records, although keep in mind that my WilsonORMapper supports paging to make the larger number of records easily manageable. I also added a new GetDataSet method that returns DataSets and performs just as good.

Why does my O/R mapper still perform a little slower than DataSets with many records? No matter what I did, almost every millisecond could be attributed to the fact that my mapping framework stores a second copy of each value in the manager for its state. This state allows you to check if the values of any entity object has been updated, as well as giving you the ability to actually cancel all the changes made to an object. I may also someday use these extra state values for more specific dynamic sql updates. On the other hand, large DataSets load faster initially since they don't load twice, but they also may have larger overhead in the long run as they track all later changes. DataSets also have a considerably larger serialization when they are remoted, so you should also consider this additional overhead that occurs in distributed environments.

What did I do, other than implementing the IObjectHelper interface, for performance? The biggest performance change I made, far bigger than reflection, was changing all of my foreach loops over hashtables to instead be regular for/next loops over typed arrays. The next biggest performance gain was changing a hashtable key from a struct to a string, which could not be just a regular object since each object instance was always different. Next, and still making a slightly better performance impact than private reflection, was accessing each datareader value only one time, even though I had to store it twice. I also now save the reflected FieldInfo, for the cases when reflection is still used, which did make a small but measureable difference, contrary to Steve Eichert's report. And of course, you can now implement the IObjectHelper interface to avoid reflection.

I also made a few other observations in the course of all this performance testing. Most surprising to me was that I could not find any significant difference between accessing datareader fields by index or by name -- it was barely measureable at all. I also confirmed that there was no measureable difference between sql and stored procs. Next, while Steve Maine noted the huge differences that private reflection can make, it is still a relatively small part of the bigger picture that includes data access. This is in agreement with several comments I received that there is a whole lot more going on in an O/R mapping framework than just worrying about how the values are set. Also note that public and protected field reflection was hardly noticeable in tests. But overall, the little things like foreach and boxing were the worst offenders.

So if you were first concerned that my WilsonORMapper didn't have adequate performance, then think again and download the demo for yourself (the demo only runs in the debugger).

# Records 1 100 10,000
# Repetitions 10,000 100 1

DataReader 1.91 0.14 0.11
DataSet 3.69 0.21 0.21
OR Direct 2.29 0.31 0.37
OR Reflect 2.78 0.75 0.81

MasterPages Template Properties in .NET v1.*

I've been asked one question about my MasterPages for .NET v1.* often enough that I suppose I should blog about it for everyone's benefit.  The question is how do you access public properties that have been exposed by the template user control from your pages that implement MasterPages.  This will be very easy in .NET v2.0, since MasterPages will contain a Master property in the Page class that will automatically be strongly typed to your master. But how do you do it today, in .NET v1.1?

C# Version:

private MyTemplate Master {
  get { return (MyTemplate) FindControl("MyMaster_Template"); }
}

VB.NET Version:

Private ReadOnly Property Master() As MyTemplate
  Get
    Return CType(FindControl("MyMaster_Template"), MyTemplate)
  End Get
End Property

Assumptions:

The user control that contains the template for MasterPages is of type MyTemplate.
The MasterPage control itself is assigned the id "MyMaster" in the page using it.

Extensions:

The Master property can be added to a base page class if the "MyMaster" id is consistent.
The Master property can work with several templates if there is a common base user control.

Posted by PaulWilson | 6 comment(s)
More Posts Next page »