Solid Architecture for Quality ASP.NET AJAX
I've written a fully fledged asp.net 1.1 CMS system with a lot of the features found in asp.net 2.0 such as navigation controls, personilazation, webparts framework (we call it page modules) e.t.c. If you are intressted in seeing anything of that, contact me.
Welcome Dino, you are one of my favourite authors on ASP.NET, I'm really looking forward to see how you cover ASP.NET 2.0.
Welcome aboard Dino! +1 on what Scott Galloway said (+winforms) :-)
Just getting really stuck into the winforms stuff...wasn't dissing your Windows Forms articles...just don't know them as well :-)
>>Just getting really stuck into the winforms stuff...wasn't dissing your Windows >>Forms articles...just don't know them as well :-) I'm going to have a *great* (sorry for being modest <g>) WinForms article published on MSDN Magazine early in January. It's about extending CURRENT WinForms menu with some graphics. I'll post about it soon.
Glad to see you here. If you are writing code to use viewstate, how do you make the data private? Is it just a conceptual difference?
>>If you are writing code to use viewstate, how do you make the data private? >>Is it just a conceptual difference? Sort of. Private viewstate is just a property implemented through the viewstate but private or protected. For example: private int _sortDirection { get {return Parse.Int32(ViewState["SortDirection"]);) set {ViewState["SortDirection"] = value;) } With control state you use _sortDirection as a strong-typed property but avoid the intermediation of the ViewState collection to read/write it. Faster! To make a private variable survive page postbacks, you have to add it to YOUR own collection (personal ViewState property). For example: private override object SaveControlState() { object[] stateToSave = new Object[…]; stateToSave[0] = _sortDirection; stateToSave[1] = _myProperty1; : return stateToSave; } The framework will serialize and encode that object for you. It can be an array as well as any other serializable type. It will be restored in LoadControlState. private override void LoadControlState(object savedState) { object[] currentState = (object[]) savedState; if (currentState == null) return; _sortDirection = (int) currentState[0]; : } PS: Just quoting from my next Introducing ASP.NET 2.0 book :-))) PS2: Sorry in advance for any other shameless plugs :-)
Dino, this is wonderful news that you are now a blogger on weblogs.asp.net! I saw you in Atlanta at TechEd 2001 and have been a huge fan ever since. I purchased (and read!) both of your MS Press books on ADO.NET and Applied XML Programming. I hope you'll continue to find time to blog here as well as continue to write elsewhere! Welcome!
Benvenuto a bordo :)
Dino, AFAIK, in the latest bits __CONTROLSTATE has been merged into __VIEWSTATE for perfs reasons. Or are you saying they're splitted again? -Vic.
Victor, what do you mean when you say "latest" bits? I don't have the PDC or later bits, but the alpha I have is nearly identical to the PDC build. (So I was told <g>) My alpha uses two distinct hidden fields. However, merging control state and view state in a single hidden field makes sense from a perf perspective. A lot of internal code is shared between viewstate and control state management. Merging them together is a good idea. --Dino
The Image points to localhost
Ooops. Fixed.
Very small image... cannot see details.... :( Why do you don't use the CSS property TEXT-OVERFLOW?? CIAO Michael
Ever thought of using CSS for this, and reducing load on the server? :) http://msdn.microsoft.com/workshop/author/dhtml/reference/properties/textoverflow.asp (works only in IE, i think)
Text-Overflow is a certainly an option. However, it is not supported by all browsers (though this is probably a minor drawback now). In addition, it only adds an ellipsis. In addition, it doesn't give you a tooltip. My point was just having a tooltip ONLY if necessary.
"All in all, the good news about Whidbey (and the MS trend) is that with the same effort you can now (actually, in 2005)" ... does that mean Whidbey will be released in 2005? :) (Yesterday I saw a presentation about asp.net 2.0, completely without writing any code... amazing :) (not always what you want of course, all those queries in the html, but still..) Your presentation earlier in november in Delft/Netherlands was very good :)
The quoted stats say, I believe, that ASP.NET apps could be rewritten with only about 30-40% of the code that is currently needed. This would naively suggest that the next generation of apps would be smaller. However, apps are seldom just rewritten, so I suspect that they will be just as big, but we'll be writing different code instead. Code which adds new or more powerful features, which in turn will probably be achievable via codeless techniques in the future. I believe this is "a good thing" as users get better products, app developers get more interesting jobs to do, and language tool development (IDEs, compilers, etc.) continue to have more to offer with each new release.
Dewayne Mikkelson and his Radio WebDog, Shadow
I think it will make things much easier for begginers, but the people who are good with .NET now will continue to know how it all works under the hood, which is very important to know when you want to do something that the controls and frameworks were not designed to do, we can have codeless programming, as long as we onl ever want to create applications that the frameworks will support ... otherwise it's going to be the same code that we're writing now. As for the code reduction I think it's more like 70% if you take in to account the loss of control declarations etc.
I agree with you that the codeless scenarios aren't necessarily the most important selling point of ASP.NET Whidbey. But they do make great demos, and they do particularly highlight the extent to which code *reduction* (as others have already noted) is a big theme of Whidbey. Also very important to me as a developer is the removal of vast amounts of junk code that is needed solely by the IDE for maintaining the current brittle code-behind model. Having code in pages and code-behind modules that was not (and in fact should not) be written by or modified by the developer was a poor design decision that I'm very glad to see being reversed.
Not wishing to be picky or anything, Phil, but rewriting with 30% of the code is a 70% reduction!
A recent statistic I read (can remember source right now) stated that less than 20% of code written every year is "new" code. Why is that? It's because software developers like you and I always start off with the assumption that we can do a better job if we write it ourselves (which in a very small number of cases is actually true). Take for example the ASP.Net datagrid ... you still have to write a crap-load of code in order for this thing to be as functional as the datagrid that shipped with VB 3.0. It amazes we that we are still writing code from scratch for things whose requirements have changed very little over the past 5-10 years. I say if we focused as much effort on creating useful applications as we did on re-implementing (or re-inventing) the wheel, we would have some killer "apps" out there. But that's just my opinion :-)
I think that one of the down sides of the code reduction is that, absent source for the framework, there are lots of times when you simply cannot debug a portion of the applicaiton. Lets say to have done lots of stuff using declaritive code. And you declare something incorrectly. How do you debug that, presuming what you have declared is not so invalid that the object created byt eh declaritive code throws an obvious exception? This is where I think the need for source code for at least the supporting code is essential. For example, you can often do a lot of things in Delphi just using non-visual components (meaning, datasource controls, query controls, etc.). Not quite declaritive, because underneath it all, there is code being written, but it is transparent to most developers. If in fact you have a complex set of relationships between various non-visual controls, there is in fact none of your code you can debug. This is where having source (in the case of Delphi, the VCL Code that acts as a wrapper around the underlying ADO) can be a life saver. It may be that what you are doing is not reasonable, it may even be a bug in the VCL, but no matter what it is, you have a better chance of finding it.
Welcome, Dino!
Welcome Dino! I've been wondering when you would start blogging. I've noticed that you started to talk a bit about Avalon in the magazines, and as a fellow Element Behaviors developer, it seems to be a natural progression. Now it is up to us to bring the rest of the world up to speed via out blogging. I don't know if you want to blog about the Avalon stuff here, or have Robert create an account for you on Longhornblogs.com DonXML Demsak
I'm going to get both books for Christmas :-) Dino, I love your articles and books... Every single one of them! Why do I like your writing so much? Because, like Carl Franklin mentioned on .Net Rocks you always seem to address real world problems and provide real world solutions, not just snippets of "hobby code". Both of you keep up the tremendous work. I aspire to be just like both of you, if that is possible :-)
One thing that is really irratating me about ASP.Net 2.0 is that all that declarative programming is being called codeless. A line of declarative programming is a line of programming. I've seen no app written with no lines of code and I've seen similar demos claiming that, but there are still lines of code! Even if the IDE is writing the code, it's still being written. I think the move to more declarative code, especially when the code it's replacing is very repetitious, is a great thing, but there is no need to lie and say that there is no code at all.
Hi Dino, Yup, that's my article on personalization all right. Which event were you at? I'm sorry we didn't get a chance to meet, as I would really like that. As for the meat of your post, I absolutely agree that it is a disservice to our audience to focus solely on the declarative aspects of Whidbey, and the code reduction this model allows, to the exclusion of other, in some cases more powerful, new features. For example, the new provider model that underlies many of the new features such as personalization, membership, etc. allow ASP.NET to support some very rich functionality while still providing developers with a great deal of flexibility in terms of the back-end data store used to support that functionality. I also think that Rob Howard's efforts to get the community involved in writing new providers will be very fruitful for the developer community (not to mention educational for those involved in the project). I also think that an important distinction that needs to be made in terms of code reduction is just *which* code is being eliminated. I think most developers would agree that writing boilerplate code over and over again for common tasks is not terribly efficient or enjoyable (which is precisely why we use and love server controls, since they allow us to encapsulate and reuse code that would otherwise have to be written over and over again). Whether used declaratively or programmatically, the new controls in ASP.NET Whidbey aren't just about reducing the total lines of code one must write, they're about making it less necessary to write code for common situations that one now writes code for repeatedly. As for cool demos vs. emphasizing the power behind them, I think that doing the "codeless" demo, and then delving into the power behind the demo could also be effective, but I agree with you completely that just doing the codeless demo is probably not enough in most cases (the exception, in my mind, being when one is giving a high-level overview, in which going into detail on the datasource controls would require sacrificing coverage of other important features). I do appreciate the recommendation of my book. I have long been an admirer of the quality of your writing, though I wonder where you find the time to write as much as you do and still teach and take on other tasks. :-) Jim, I appreciate the kind words, but while I can't speak for Dino, I assure you there's nothing all that special about what I do. I'm just an enthusiast about new technologies who's fortunate enough to get paid to play with new toys and tell people about them. ;-)
I'm talking about the M3 bits. We can assume they're still merged then ;-) -Vic.
I like both of your posts guys. I am developer and trainer myself. I am glad that you started the post on Whidbey and concern about presenting the new powerful feature set to the developer. My first look at several articles published on MSND site such as “Bind Your Data in ASP.NET Whidbey” gave me a bitter taste. It is good that you do not have to write code for data binding but is it the best practice? Unfortunately developers take the code from article as such and make production level application. I really liked MVC model and I want to see more articles that support that point of you with new features added. My two cents, Maxim
Alright, then build a simple client-side Javascript to implement this. I see no reason why this job has to get done by the server :)
Nice idea, nice implementation. And nice to see someone intelligent make HTML a little more intellegent!
I've never been happy (in my 7 years experience professionaly, and 15 all together) with tools that do the job for you. Sometimes they have value. But overall, as a rules of thumb, the more I end up "relying" on some tool to produce x% of code for me I always have ended up needing to scrap it for some limitation at some future point of development. Perhaps 2.0 can greatly assist. 1.0 was great. Businesses will love it. Especially consulting houese. But I just can't see myself embracing it as a rules of thumb. Perhaps to compliment some of our work but that's about it. I can't see myself complimenting its work but in reality, they are designed for certain purposes and my business needs/requirements have always been of a different purpose. While I've had many argue me fervently that I'll be out of a job while they are making applications, well, hate to say it this way, but, the level of productivity I enjoy is competitive with that of my peers. The point is that I know my stuff. I know how things work. I know what the problem is when it doesn't work. I know why things work well in this case and not in that. I understand win32, I understand .NET, I understand COM from an internal perspective. Put all this together and I know how to design, implement, debug, and maintain enterprise applications. I seem to be equally good in all those areas. Relying on code generators and things that do my job for me only tips the balance, although I'm not narrow minded. If it'll help me get to point B I might give it a try. Just wait until I have a complicated requirement that the "automatic" stuff can't handle "easily" and it'll go out the window. Thanks, Shawn
Dino, there's an EXCELLENT article in last month's issue of the MIT technology review regarding some pioneers of this area. The article can be found here - > http://www.techreview.com/articles/tristram1103.asp
I forgot to mention on my post above that your blog entries are quite informative. Keep it up =)) Thanks, Shawn
Hi Dino, My suggestion for a feature to tackle is XHTML output from .NET 1.1. My company is probably switching to the .NET platform for our website and we would like our output to be at least XHTML 1.0 Transitional, maybe even 1.1 strict. We can't wait for Whidbey, so we have to find a way to do this in .NET 1.1. I'm sure lots of other folks would be interested in this topic, as web standards are becomming more important. I Look forward to your articles, Steve
How could I get this to work where we set the style using CssClass and a .css file. I can't find the correct FontInfo object as the ItemStyle doesn't get populated from the .css file. I don't want to hardcode the style for my grids (other than width).
Can you use ExtenderProviders to extend Web Controls? For example, could you do the same thing for System.Web.UI.WebControls.TextBox? The ExtenderProvider would, of course, have to render some client-side JavaScript. I tried something like this a while back and couldn't figure out how to get the new Properties to be persisted in the aspx file (or the CodeBehind).
O2003 rocks. It catches approx 98%-99% of my daily spam dose (~150 mails) with only one business-related false positive since the beta days. (i.e. "one error in 200 days" or such) It however also catched some newsletters which is not exactly a problem for me but might be for others ... -Ingo
Dino, It is always hard not to give away much of a free stuff. It looks like you always were able to manage a good balance. I was wondering how you are going to use Blog :). Maxim
Hi Robert, extender providers are only supported on Windows forms. I can't figure out myself a special architectural reason for it not to be supported by the ASP.NET designer. If somebody on the ASP.NET team is reading this, we'd love to know more.
Is this the same IE-only method they printed last Winter/Spring?
Glad to see you talk about this. It was a kind of 'strong' arguments to say the least few weeks ago in this weblogs. I use myself an htc file that I found through MSN. Suit well my needs because it's for an admin tool, so IE obviously. I heard this scrolling story will be fix in .Net 2.0. Can you confirm ?
By the way Dino this article is not supposed to be the first part. What's about the second part, where you have the headers fixed whatever the width size of the columns ?
Outlook 2003 + SpamBayes (http://spambayes.sourceforge.net/) works wonderful. After training SpamBayes on spam vs real mail for a while, it gets just about every spam I receive and not a single real mail.
I guess you have to be a subscriber to get the source? Anyway the source can be made available with this requirement?
> Is this the same IE-only method they printed last Winter/Spring? The method has a dependency on the overflow attribute. IE supports it since version 4.0 I believe. I'm pretty sure that latest Netscape browsers will accept it as well. If you refer to the article(s), I was talking about part 2 which differs from part 1 mostly because I have packed everything into a custom DataGrid control. Part 1 used an ASCX with an embedded grid. This had the drawback that you must modify the source to bind to different columns.
> I heard this scrolling story will be fix in .Net 2.0. Can you confirm ? What I know about version 2.0 and scrolling is that the Panel control (i.e., the <div> tag) will have a scrollable attribute. The implementation is obvious--they just turn the CSS overflow attribute on. If you're referring to something specific of the DataGrid, well I just don't know. If you do that using an HTC, then you can more easily exploit the clientWidth property (think that is the name) which is the only method I know to get the REAL width of a table column in the browser. I realized about that after writing the article.
>I guess you have to be a subscriber to get the source? Anyway the >source can be made available with this requirement? I believe so. However, I'll have the source code free on the www.CodeArchitects.com web site starting the 1st of January. In general, one month (approx) after the publication of the magazine. You need to get a free registration. For details, see http://weblogs.asp.net/despos/posts/42776.aspx
The sad thing is other browsers have supported overflow on the TBODY for several years (per the HTML 4 spec) & IE still doesn't. That technique should easily combine with minimal (IE-only) CSS like this: http://slingfive.com/demos/scrollTable/ I think both could be applied to the Datagrid after reaching the browser, via some smart use of CSS selectors.
Here in Hawaii, Spam is a delicacy... Nothing like SPAM Sushi!!
I'm sitting here at 5AM in the cold Windy City thinking exactly the same thing. Regarding the kids, think "red-eye" - no point flying while the sun's up. Merry Christmas!
You're welcome in Iceland anytime, but I definately recommend coming here in the summertime :) Merry Christmas from Iceland!
This is most interesting! Is it possible to get a draft copy of your article.. too impatient to wait for few months for CODE mag..? :-)
Security and snot...what a *delicate* description. ;-) Definitely worth a chuckle (not to be confused with a cacls).
I implemented a "PersistentStorage" feature in v1.x like a year ago for a project that needed it. It works/support pretty much the same as personalization in v2.0 but of course without all the fancy things like a strongly-typed profile class, etc. Looking fwd to see you code to compare it :-)
Cacls.exe works great when you're deploying to a local machine or one that you've got privilieges to run command line programs on but... What mechanism, other than an administrator, could you use when deploying a site over FTP?
Well, we've only got 4 feet of snow up here on the Green Mountains of Vermont, but you are always welcome here as well!
>Cacls.exe works great when you're deploying to a local machine or one >that you've got privilieges to run command line programs on but... >What mechanism, other than an administrator, could you use when >deploying a site over FTP? Good point. I've never personally faced FTP deployment. Sorry... The only trick that sounds reliable in this case is resorting to the Win32 API, arrange a custom action within the MSI and go. But if you're only allowed to copy files I'm afraid you can't do much more than ask the sysadmin to do that for you. Again, I'm not the Win32 security API super-expert but I don't think that ACLs got copied from machine to machine and over FTP. But even if they could, what if you need to grant that permission to a folder for the app to create dynamic files? As far as I know, though, in the real world admins normally allow for a writable folder pre-configured for the running version of ASP.NET. In this case, you just drop any writable files there. Admittedly, this solves the problem only if you need to enable ASP.NET to write/edit a file; not if you need an ACL for certain users. The more I think, the more I feel confident that asking is the only *safe* way to go. Am I wrong?
Funny you bring this up. I researched this a few weeks ago as well and ended up doing exactly this - adding a call to Cacls from the post setup configurable (which does things like create the virtuals and customize the initial Web.Config). The only problem with this is always that hte user has to have rights to use the tool in the first place (I think it's admin rights). I see this more useful for the situation where you distribute an app. You need some way to automate the setup so that the user doesn't have to create virtuals and set permissions. As part of this step you need to set the ACL to allow anonymous access. You also might need to set rights for the Admin directory allowing only special users into that dir. You definitely want to prompt users before making these changes automatically, but of course you *could* do it automatically which is kind of a scary thought.
Hey Dino! I'm waiting for your visit to Alaska! We don't have 50 inches of snow on the ground in Fairbanks yet, but we're pretty close to mountains that do. Get over here!!! You always have a place to stay. Don
first i wanted to say i am glad you decided to start a blog, i've been reading your articles/books for quite some time now and they have always been extremely helpful, if not usually over my head. this post got me to thinking, so i blogged my own reply here (won't rehash it) http://zerotrilogy.gotdns.com/jaysonblog/. your post reminded me of this article http://www.ftponline.com/wss/2002_11/online/tmurphy/default_pf.aspx, thus i believe it's safe to assume that microsoft will provide many different solutions to obtain a common goal so that all means of programmers can get there. i have my own personal beliefs on xaml, but won't waste precious space with them here :-) please keep the posts coming!
I was a bit confused by the "title=GetTheString();" I guess this should be "title=e.Item.Cells[1].Text.Trim();" Also the "return String.Format(fmt,text,rawText)" is unnecessary as a "return rawText;" will do the same. Finally in the itembound code I added: // Set the text including the tooltip when necessary e.Item.Cells[1].Text = newText; if(newText!=title) { e.Item.Cells[1].ToolTip=title; } This actually set the tootip text, this then works most of the time but trimming 3 extra characters is safer than 2 when adding the ellipsis.
I've been experimenting with this one myself over the last few days (and .ASIX files as well). Truly very cool. One thing though - for generated images I've been resorting to using .png as the output format because for .jpeg the image quality is pretty horrible...after some experimentation I've come to the conclusion that the limitation is at the DynamicImage end rather than the ImageGenerator end. There must be something I can do here that I'm missing...any idea how to control the quality of a generated image from the DynamicImage control? (the observed effect is very bad colour bleeding and ghosting - unfortunately all the published examples I've seen use a black background, which coincidentally hides the problem to some extent). I'd be very grateful to know if there's a known way to fix this.
Excellent article Dino, I wonder if all those features could have been given out as part of the Datagrid control itself. but alas there isnt a complete / perfect world isnt ;)
There's a new API for managing ACL's in Whidbey. Look in System.Security.AccessControl.
See next post
It's a fact, the JPG codec that .NET uses is NOT good. I've done quite a bit of testing with this, including setting the quality like you mentioned, but even if I set it WAY high it won't even compare to 50% quality in Photoshop's JPG codec. Or could this be a bug?
I've used this method to manipulate jpeg quality in the past with .NET (most significantly in an image manipulation program, where both resolution and quality were adjustable with sliders),and it's always worked fine for me. So far however I'm not having any luck doing it with a .asix file (I'm overriding CreateImage, saving to a MemoryStream and reloading the image from there)... I get the same result regardless of specified quality, and the effect is very noticeable (if you want to see crappy jpeg quality, green text on a grey background is always fun). Tomorrow I'm going to dredge through all my old bits of code to see if I've missed something.
>>So far however I'm not having any luck doing it with a .asix file (I'm overriding >>CreateImage, saving to a MemoryStream and reloading the image from there)... >>I get the same result regardless of specified quality, and the effect is very >>noticeable (if you want to see crappy jpeg quality, green text on a grey >>background is always fun). Tomorrow I'm going to dredge through all my old bits >>of code to see if I've missed something. This can be anyway due to a bug in the alpha, though. I have no problem instead buying the statement the .NET JPEG codec is ... not that great. So to speak.
Well, after doing a lot of tests I've found that Whidbey is actually consistent with the behaviour in 1.1...where I was getting results was because I was using a completely dynamically generated image. It appears that if the original source of the image is an existing image, the quality settings have about the effect you would expect (which is what I observed with my image processing app). If however the image is generated from scratch, any setting other than a very high one yields pretty horrible results (and it's never all that great). Strange, but true. Anyway, if I need to create an image from scratch I can probably live with PNG (which is excellent).
Here's how I reorder columns in a custom DataGrid. I have a custom DataGrid which selectively renders columns in a user-defined order: - Persist the column order (column separated list) somewhere appropriate depending on how long you want it to persist (ViewState, Session, Cookie or server-side database). - Don't change the column order of the DataGrid. - Override RenderChildren to render the columns in the required order. When RenderChildren is called, the controls collection will contain a Table, which in turn contains a collection of DataGridItems. So the code looks something like the following (actually a bit more complex: e.g. I always render in the standard order when in design mode). protected override void RenderChildren(HtmlTextWriter writer) { Table table = Controls[0] as Table; if (table == null) { base.RenderChildren (writer); return; } table.RenderBeginTag(writer); foreach (Control ctl in table.Controls) { DataGridItem dgi = ctl as DataGridItem; if (dgi == null) { ctl.RenderControl(writer); continue; } if (dgi.ItemType == ListItemType.Pager) { dgi.RenderControl(writer); } ctl.RenderBeginTag(writer); // tr tag // ctl's Controls collection contains a collection of cells. Render them // in the required order ... ctl.RenderEndTag(writer); // /tr tag } table.RenderEndTag(writer); } With a rendering solution like this the column order does not change on the server side: i.e. no need for rebinding, also it ensures that the control tree stays the same across postbacks.
Hi All: I had used Extender provider with web forms just fine. It is supported even thought the implementation could be a little different. See MSDN for details
I just checked www.CodeArchitects.com and the code is not yet available... Have you change your mind about making it available ???
Dino, You mentioned in another post that you hoped to get the source code for the Scrollable Grid up onto www.codearchitects.com from start of January but it doesn't seem to be there...any chance this will be happening soon. Many thanks.
Dino, there's a hotfix which makes this change in ASP.NET 1.1 - see http://support.microsoft.com/default.aspx?scid=kb;EN-US;823030
DataGrids usually render huge amounts of HTML so this process, although feasible, could end up very costly with all the text parsing. One other area to explore would be create a subclass of the TableCell control that renders TH instead of TD and during the ItemDataBound event of the grid transfer all the contents and properties/styles/attributes of each TableCell in the Header TableRow (or DataGridItem) to new instances of the new TableCell class.. Then remove the old tablecells from the row and add the new ones..
Hello Dino! Excuse me, but I cannot find code of ScrollableGrid control (aspnetPRO articles) at Code Architects Web site.
Incidentally, the hotfix package can only be obtained by calling support...why they can't just make the thing downloadable from ASP.NET is beyond me...Rob Howard - any particular reason?
Shouldn't you also put the table header into the THEAD element for accessibility? And put the contents into a TBODY?
I can't find it either ;-)
A very slick solution I must say. This worked great for my grid.
Martin Fowler compares the full blown Domain Model pattern with simpler ones, such as Table Gateway in his book Patterns of Enterprise Application Architecture. I guess as ADO.NET provides is pretty much a Table Gateway with no need for user code with strongly-typed DataSets, you might need a more complex solution before reaching "break-even" with an ORM. You might also want to check out NEO (now at http://neo.codehaus.org/) as an alternative ORM.
"... Your application passes objects and gets objects in and out of the data source. Any serialization/deserialization process is taken care of and application developers don't need to know about the data model and SQL details." This is a myth, Dino :) serialization/deserialization is not done. This is because you are using your own code for the classes. Therefore, if you want them to be serializable, you have to add the logic yourself. (i.e.: if you add an eventhandler, you have to implement ISerializable and if you add a member of an interface type you run into trouble with the XmlSerializer which is bad for webservices.). Developers DO have to know about the relational model. Because the developers develop the classes themselves, they have to know an order relates to customer in a m:1 way so that the 'Order' class has to be able to hold a reference to the Customer class and the Customer class has to hold a reference of a collection meant for Order classes. Furthermore, if you want to filter on objects, it is very useful that you understand which objects are mapped to which tables and how these are related so you can formulate a good filter (query) which will actually be what you want in SQL. This is impossible without knowing the relationships in the model. "...What's the real cost of this code? For Each c As Customer in reader ' ... Next" The real costs are not in that kind of code. The real costs are in the tracking code of objects. Because you are using your own classes in Objectspaces, it has to do either one of the following things to keep track of changes of properties: 1) keep a copy of the original object so it can check which fields are changed (slow) 2) alter the byte code of the object at runtime so it can insert event handler calls into the memberset code in the properties (dirty) 3) do not check for changes but assume that ALL fields are changed 1) is slow because it has to verify each property when you want to save an entity/update an entity. This can be cumbersome if the entity contains for example an image or a big blob of text. In the Java world this is seen as the wrong way of doing things. 2) is faster but because of the runtime bytecode changes, it is IMHO a dirty way of doing things which can lead to possible problems with own code. 3) is also slow, because you execute a lot of extra SQL in a lot of situations (imagine you update a single field in a 20 field table) When I checked the SQL generated from Objectspaces for an update query I saw that they opted for 3) as it seems. Another solution would be 4) use a base class for the entity classes which takes care of the plumbing code like fields and other code for housekeeping like tracking changes. Personally, but I'm biased ;), I never would have opted for the non-intrusive way of doing O/R mapping (they could have opted for an interface to be implemented) because it will bring a lot of problems for the user of such a tool: how to support complex databinding for the customer.Orders collection? -> you don't want to let the user edit PK fields, and other fields which are read-only. This requires ITypedList and property descriptors. Not a nice thing to do :) (and I haven't talked about IBindingList :))
How complex is my ORM? Mine is as simple as I can get away with, and unfortunately, that's not very simple in terms of the actual ORM process itself. My perspective is code generation with the base model of abstracting the business information (business metadata) away from the technology (templates and components). From the perspective, I anticipate the business model having a very long lifetime - potentially more than a decade. The business metadata is an abstract view of the business and its data. To maintain the viability of this across the massive technology changes a decade can bring requires abstraction. Pragmatically this is an abstraction layer between the database and the business objects. That’s the job of ORM. The ORM is a pluggable layer that can start out simply allowing mapping. With current technology, ORM can be partially or wholly implemented in stored procedures - which can be generated from the ORM model, along with the corresponding extendible plumbing business objects. I think people that don’t perceive themselves as using ORM are actually using this sort of simplified version in their stored procedures and logic to load their business objects. You at least control the columns and children returned. The goal for my ORM is to be only as complex as it needs to be to raise this mapping from being encoded as technology to being a true abstraction layer. This explicit abstraction offers the possibility of a much more complex mapping in the future - such as across diverse storage technologies. We need this ability to map during development to accommodate future change, and we need it to have no impact, or a positive impact on other quality issues such as runtime performance. That brings us back to ObjectSpaces. When I saw ObjectSpaces at PDC, I was under the impression that it was fairly slow, especially compared with managing your own business objects.
I use our own product (the well known EntityBroker) right now to write a Content Management System. A real one, for all thinking dotnetnuke is a CMS. I love it. The trick with them is oyu need to THINK in objects. Using object inheritance to control baehavior - for the content store we use a pretty wide and deep hierarchy of "containers" (for files, for images, for documents, for templates), and the O/R mapper (NOT ORM - damn, get your acronyms straight) allows me as developer writing code to use OO for programming. The correct subclass is determined automatically by the mapper, and I use virtual ethods plus attributes to control behavior (like every container has attributes telling it under which container it is allowed - you can not put a folder under a file). Also, the stuff I do works against SQL Server, Oracle and Access out of the box. No changes required. I must say, though, that I am trying to push our O/R mapper to the limits - I use a winform application for content editing (which is SO much better than the regular crap approaches using html based editing - guys, think about the performacne, local help etc.), and I use our remote DataServer access to hide the database connection from the client (all calls are routed through IIS - about 5 lines of code) The trick really is - thingsget better the mor ecomplex your application gets :-) And - naturally - the more you do know about OO.
I use my own, LLBLGen Pro, in several projects (one being a CMS) In 2002 I wrote LLBLGen 1.x, which was a stored proc + class generator for sqlserver, and it did the job, but was limited still, you still had to fight with datatables etc. After I finished LLBLGen Pro it's a breeze, I never write stored procs anymore nor do I have to deal with datatables and mistyped column names in gui's... It's so simple now. :). A few lines of code to search, oh, a new filter has to be added: 1 line of code. As Thomas also said: multi-database support is a big win. We too abstract away database dependency, so you can write classes once and persist them to multiple databases, read from oracle, write in sqlserver etc. You can even port your BL code from sqlserver to oracle without any hassle. Btw, ORM means Object Role Modelling, which is Halpin's method to design relational models in an abstract way (very cool, check out: www.orm.net ). MS even offers a free tool to model your db in: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=27FE6786-A439-4286-B8B6-7A9B84CFA709
We use a O/R Mapper we wrote in house for most of our projects. I'm definitely a fan. The ability to write a business object and let the mapper "generate" the data access is a major boost to productivity. There are some situations where I don't think they work, however, for the majority of business applications I think they fit nicely.
I used a dirty flag in my base object to track whether or not the object has been changed. It does introduce a little bit of extra work for the developer. However, the extra SetDirty(); call in property sets isn't too big of a burden. I do like the idea of letting the framework keep track of all the properties and whether or not they've been changed but as you said I've stayed away from it because of performance. Based on some recent posts from Paul Wilson I'm wondering if its really much of an issue in the overall perf of the entire app. I've put it on my list of things to look into for future revs of our mapper.
I'm using my own WilsonORMapper (http://weblogs.asp.net/pwilson/archive/2004/01/07/48405.aspx), which is based on the syntax of ObjectSpaces. Its not as advanced as some of the others, but its simple to use, very performant, and it works for me. I've got several other posts on my blog about why I choose to not use base classes or attributes, and also some comments about how I track changes -- all of which is very much just like ObjectSpaces.
It's how you track the changes, Steve :) In the situation of ObjectSpaces, if I have an own written class 'Customer' and I change the property 'CustomerID' somewhere in my code, the Objectspace object has to know that I did that. This means that it has to track changes somehow but because I wrote the whole class, it's not possible to add an event handler somewhere :) This thus opens up some problems: keep an old copy around or inject code in the runtime byte stream?
ObjectSpaces tracks changes by implementing the IObjectNotification interface... by hooking event handlers that is. http://longhorn.msdn.microsoft.com/lhsdk/ref/ns/system.data.objectspaces/i/iobjectnotification/iobjectnotification.aspx
Tracking changes is something else than simply act on what you're told to do I think. If you've changed 1 field in an entity with 20 fields in a collection with 100 other entities, you don't want to walk all 2000 fields and check whether they're changed, nor do you want to execute 200 update queries with 20 set clauses if you just have to update 1 field, still the user calls myCollection.Save() or myBroker.PersistChanges() and you're tracking all entities in the collection... It's then key to know which entity is 'dirty' and which isn't, then which fields are changed and which aren't. It should then result in a single update query with a single set clause. I don't know in detail how Objectspaces tracks changes under the hood (it looks like they don't, they simply act on what they're told), but if they're doing nothing more than collecting objects which might be changed, it's obviously very slow compared to more clever code.
Home made framework. One thing that really bothers me in OR-mapping solutions is the lack of round trip engineering. A sidenote: be aware of the depency between you and your OR-mapping solution vendor. I personally prefer open source initiatives.
::A sidenote: be aware of the depency between you and your OR-mapping solution ::vendor. Name me one who follows a standard. Ups. ::I personally prefer open source initiatives. This would make our commercial EntityBroker perfect for you - it is open source.
By the way, ObjectSpaces does track changes and only updates what is necessary from what I've seen in my tests.
"This would make our commercial EntityBroker perfect for you - it is open source." Since when? :) It's sourcecode is open for people who pay, or am I mistaken?
Dino, I personally think that adoption of ORM mappers are still slow and not a mainstream because every ORM is implemented based on its own Framework, so before you can be productive you need to learn framework first and then start coding. For example, Datasets objects are still not used correctly by most developers. Why? Answer is very simple, all of the codes and samples you find do not talk about what are the problems can be solved with Datasets rather they are just code-snippets here and there. What would be cool is to have an article about what does it take to write a ORM application. Presistance and ext.... My two cents, Maxim
"One thing that really bothers me in OR-mapping solutions is the lack of round trip engineering." No development tool other than perhaps XDE supports roundtrip engineering, due to the fact that somewhere in the process sourcecode is involved. A lot of O/R mappers out there do their best to support roundtrip engineering to some extend though. "A sidenote: be aware of the depency between you and your OR-mapping solution vendor. I personally prefer open source initiatives." Erm... open source initiatives also have dependencies: when you start using a tool, you depend on that tool. Open Source might sound great, but when it comes down to changes you need you have to get your hands dirty as well, with the risk of loosing any future updates. Btw, our source is open for all of our customers: code generator, task performers and runtime libraries .. oh and the templates of course ;) " By the way, ObjectSpaces does track changes and only updates what is necessary from what I've seen in my tests." Are you sure? Update SQL still contains checks for NULLs in parameters (the optional parameter trick), and supplies parameters for all field afaik. Still, it's a mistery how it determines what's changed. If it does this through code injection (or a secret CLR short cut) it might be ok (but has risks for code written by the user), if it does it by comparing with a shadow object it is really slow and will not improve a lot in the future if they're not changing that.
Maxim, Ditto++ Do a search for "Databinding ASP.NET', almost every page returned will how you databinding a DataSet to a DataGrid. It's the same with ObjectSpaces, do a search ObjectSpaces and you'll find a ton of snippets mapping "Orders" to "Customers".
Hey Frans: The update sql checks all parameters, including null checks, in the where clause for optimistic concurrency, but the update itself was more fine grained.
To further extend the list of O/R mappers for .NET I want to add Genome :-). We are using O/R mapping now for over two years in our own .NET projects. We gradually extended the functionality on the go. Our projects are large scale enterprise applications in the governmental sector. O/R mapping helped us a lot managing and parallelizing the development of these long term projects. We also do not see any performance penalties in using O/R mapping, opposed to some other opinions seen in the community. Of course there is no silver bullet where you get performance automatically, but optimizing hotspots in a well designed system is much easier and O/R mapping can help a lot in this regards. Caching strategies and greedy loading can be controlled decoupled from business logic. We can execute expressions on the client side as well as on the server side through automatically generated stored functions. As a last resort back door stored procedures can still be employed yet remaining integrated with the O/R mapping. We have created a quite extensive whitepaper that explains benefits of O/R mapping from our viewpoint, you may want to have a look. Seeing that object spaces will take yet another release of .NET and also not covering all the concepts of O/R mapping we do already we decided to make Genome available publicly. A product website will appear soon, in the meanwhile you can check out the following blog entry where you find links for downloading the whitepaper and the full functional product along with a sample. http://weblogs.asp.net/klaus.aschenbrenner/archive/2003/12/21/44983.aspx
IObjectNotification is the other way around. I'm talking about my Customer class, I tell objectspaces to track changes and I change its property CustomerID, then I call persistchanges and objectspaces can figure out that CustomerID has changed and should be changed in the database. That's hard to do if you work with non-intrusive classes.
It seems pretty frustrating to see Microsoft fail at clearly defining ObjectSpaces. Even in the Whidbey builds there's no clear vision on how this is supposed to work - the spec seems completely open and incomplete at this time. ORM is definitely something useful if done right, but like you I worry a lot about the overhead involved. It's bad enough to have business objects in an application that acts as a fairly thin veneer around your DataAccess layer - all of this adds overhead and ObjectSpaces and the like will then sit on top of this. In this respect ORM becomes a sort of framework of itself and I'm not sure that this is a good thing coming from Microsoft or other big vendor, because those solutions tend to be a bag full of things that address a very low common denominator. The big problem is that ObjectSpaces and other ORM implementations implement their own middle tier. So you're locked into a specific way of doing your data access which in many cases is sub-optimal. It's not always a bad thing, especially if you're dealing with customers who have no interested in imlementing a custom data access layer. This may or may not be a problem depending on the application, but if you go with a full ORM approach it's much harder to not use the ORM logic than it is with more traditional business object approach where you can always implement custom methods to do what you need anyway you like. In ORM for consistencies sake you will always want to go with the ORM metaphors. Ultimately I find it interesting to contemplate what the real value of a particular abstraction is. ORM is a more natural abstraction that a typical business object, but it doesn't really provide much additional functionality that you can't already provide with a typical business layer which IMHO is more flexible simply because it is a bit more low level. Sure it's nice to have an ObjectReader, but how hard is it really to read a row from a result set and have your business layer parse that into a business object data object? BTW, where did you find offical information on ObjectSpaces and its non-release status for Whidbey? Is this official or an assumption?
A lot of the functionality I would like to see in ObjectSpaces would require some changes in compiler functionality. It would be great if: a) There was syntactic sugar for injecting common code into Properties. i.e. without IL post processing. public property string Name(name); would actually be treated as something like: private string name; public string Name { get { return GetProperty( propertyof(Name), ref name ); } set { SetProperty( propertyof(Name), ref name, value ); } } where GetProperty/SetProperty are methods of an interface that objects can implement. The GetProperty/SetProperty base class implementations would then be able inform ObjectSpaces whenever a get or set on a persistable object occurs. The ref parameters allow the private fields to be get/set as needed by the ORM layer. This sort of functionality would also be useful for applications other than an ORM layer. b) Properties could be referred to in code in a typesafe manner. e.g. typeof(Person) refers to the Reflection Type instance for person. propertyof(Name) or propertyof(Person.Name) could refer to the Name property descriptor of the Person object. I would far prefer to hard code a referance to propertyof(Name) to refer a property rather than refer to it using a string "Name". The compiler could then be used to be identify code faults rather than at runtime if the string approach is used. c) Application Base class Having a fixed base class for ObjectSpace applications would be a bit of a pain. Allowing applications to have their own base classes as long as certain ObjectSpaces interfaces are implemented would be a good compromise. The application base class could then contain some helper objects which make it easy to implement the required interfaces i.e. extension by containment rather than inheritance from a fixed base class. The sort of functionality presented in (a) through (c) would allow ObjectSpaces to get into the guts of application objects without putting too much of a burden on the application programmer. In theory, the approach should result in better performance because there is less need for look ups and state comparison to process the persistable changes. The functionality would also be useful in "plumbing" type functionality not related to ORM/ObjectSpaces.
ORM=Object Role Model O/R M=Object/Relational Mapping Which did you mean?
The more and more I worked with this method I discovered its impossible to use it with viewstate. Here is why. First off you have to know in the control whether or not it is being restored from the viewstate. You want to reorder in the RenderChildren only if the data is not being restored from a postback. Why? Because you have to know how the columns are ordered on the initial databind because each time the page postbacks the grid will be restored based upon the original bind. So why can't I save the initial bind in a hidden field? Because this could change at any time say you enable sorting in which the sort function you have to rebind the data, well now your data is update and each postback from now on will use viewstate to resotre the sorted order. If there is a way to tell whether a control is being restored from its viewstate then this method will work. However, I think thats why microsoft chose to make you rebind when you use server side sorting but I may be wrong..
Oops. Sorry about that. It's definitely O/R Mapping
Dino, Setting the the 'height' property of ItemStyle of the datagrid to a constant value resolves this issue.
first of all thank you very much for a nice article. I found some issues in the current implementation. I need server-side sorting, paging and client-side drag&drop column reordering. I decided to follow your approach and it works fine, until I apply formatting. For example, on my grid, some ROWS are formatted as % values, other few rows formatted in thousands, numbers and some Columns are have no formatting (text columns). 1. When I drag-drop a numeric data column in the place of text column and do a post-back, I loose the formatting. The numeric column now follows the formatting of the original column (resides on that position before). (I apply formatting at run-time, by attaching a event handler to 'ItemDataBound' event.) 2. When I click the header of a particular column for sorting(server-side), after the reorder, the post-back sorts on different field. (i.e) Lets say originally when the page renders, the column order is A B C D E. I reordered on client side like, A B C E D. Now I click the header of 'E', means I wants to sort the data as per the column 'E'. But after that post-back the data was sorted on 'D'. I would appreciate if you could provide some additional thoughts regarding this. Thank you with regards vasutha
Sure. But I was just trying to avoid that :-) I was looking for a solution that works in a rather automatic way.
I'm aware of problems on client-side sorting. I think I've written that sorting is a pure text sorting completely unaware of types involved and as such impacted by formatting. I could be wrong but I tested it with static formatting and as long as d&d is concerned, I'd say it works. You mention you apply dynamic formatting through ItemDataBound. Don't know. I didn't try that. Apologies
I encapsulated the work I've done on mapping and code generation in a book that's just been released. The goal of the book is to show how code gen and OR/M work, both high level priciples and the nitty gritty of one approach, along with tools and templates to make it hppen. I have egg on my face because, like Dino, I misused the phrase ORM, but if you forgive me that... Anyway, I agree that establishing a permanent relationship with any vendor for how your application is actually developed is too high risk. The solution is to have control over the templates (using the term in the broadest sense), and standards to allow interchange of templates (or of metadata, depending on your perspective). OpenSource is also a relationship. Although any relationship with source code is better than any without, source code. My goal is to help demystify the process. There is no magic behind the curtain. There is work, neither OR/M, nor the templates that express it in code is easy to do. However, its quite accessible to intermediate and advanced programmers, and understanding one way the stuff can be bolted together is good for folks considering writing there own, or evaluating others solutions.
FWIW in http://www.microsoft.com/belux/nl/msdn/community/columns/jtielens/objectspaces.mspx the claim is made that ObjectSpaces are part of Whidbey.
Oh Frans :-) ::"This would make our commercial EntityBroker perfect for you - it is open source." ::Since when? :) It's sourcecode is open for people who pay, or am I mistaken? Right. Let's hav a look at the dictionary for what open source means (not what the free software foundation tries to make out of it). It means possible access to the source code. Free Software is something different.
Frans, Just a wild guess; Serialize the object being tracked into a temp. location. Then on persisting serialize the object again, and compare the binaries with the previous serialized data. Dont know how it will perform, but it is very transparent.
Since you ask, yes, I have worked on a couple of applications that were MTS based, but had to drop MTS in order to perform well enough. Actually in one case, it was sufficient for them to stop using the MTS transaction support and start managing their own transactions. That was enough to get the performance boost they required. (They didn't actually need distributed transactions. And it's a shame that MTS doesn't support TX flow for local TXs...) But in the other case, their performance was hammered by having gone down the whole stateless design in the first place. The database just wasn't able to cope with vast amount of extra traffic that a stateless design with highly encapsulated objects will tend to lead to. (And since most of the data these stateless objects retrieved each time they were instantiated only actually changed once a day, it was entirely unnecessary traffic.)
You'll accept a 30% or greater perf hit for simplicity in programming? Sorry, but I think that's nuts. Not because simplicity in programming isn't valuable - its tremendously valuable - but becuase its unnecessary. ObjectSpaces, in the last incarnation I saw, was based on dynamic programming - it did extra stuff at runtime. But for almost all of us mapping is an issue resolvable at design time. Thus I think a code generation is a much better model for most of us. I don't claim to have gotten to the bottom of ObjectSpaces, but code generation also seems to offer more flexible models for extending business objects - which I think is a critical element. I'd love to hear what benefits you see in doing extra stuff at runtime.
If I read a review on a book like that, it'd make me actually want to buy the book. There are way too many books that are more introductory than those that cover advanced topics.
Hi Scott way to get it then :-))) http://www.amazon.com/exec/obidos/ASIN/0735619034
A runtime cost of +30%. When I first heard that I was shocked. I would really like to know what it means. Its normal to be slower than SQL direct, but is there an aspect of custom code/buisness objects that needs to be factored in? I use the Data Access Application Block and SQL Server. It is fast and it is really easy to use, can ObjectSpaces truly simplify my life more than this? V Intedev had design time controls. VS.Net lets you drag datasets and connections onto the design surface. Both are/were pretty much just junk.
I don't understand the argument Kathleen is using. Although the mappings are done at design time, they are hardly an issue at runtime. What's the biggest bottleneck in O/R mappers is the generation of SQL. The problem is: you can't do that up front. You have to build in some sort of SQL-JIT which caches often used query constructs a