August 2006 - Posts

Part 1: Dynamic vs. Static
Part 2: Creating Dynamic Controls
Part 3: Adding Dynamic Controls to the Control Tree
Part 4: Because you don't know what to render at design time 

PART 2

Creating Dynamic Controls

Creating a dynamic control isn't just about "newing" one up. There are several different types of dynamic controls. There are also several different ways that dynamic controls can manifest themselves in your page's control tree, sometimes when you don't even realize it. As promised, understanding this will not only increase your understanding of dynamic controls, but of ASP.NET in general.

A Dynamic Control can be a:
  1. Server Control
  2. User Control
  3. Parsed Control

But no matter what kind of control it is (dynamic or otherwise), they all share the same base class: System.Web.UI.Control. All built-in controls, custom server controls, and user controls share this base class. Even the System.Web.UI.Page class derives from it.

This section will focus very specifically on how dynamic controls are created. So the following code samples aren't really complete. Just creating a control dynamically isn't enough to get it participating in the page life cycle because it must be added to the control tree. But I like focusing in one thing at a time. Fully understanding something makes it easier to understand something that builds on it, because you can make assumptions and focus on the new functionality.

 

Dynamically creating Server Controls

TextBox tb = new TextBox();

There's not much to them. All the built-in controls are Server Controls, and so are any controls you create yourself that inherit from Control, WebControl, CompositeControl, etc.

 

Dynamically creating User Controls

First lets create a hypothetical user control:

<MyUserControl.ascx>
<%@ Control Language="C#" AutoEventWireup="true"
CodeFile="MyUserControl.ascx.cs" Inherits="MyUserControl" %>
Your name: <asp:TextBox ID="txtName" runat="server" Text="Enter your name" />

A user control is not unlike a regular ASPX webform in that the ASCX contains markup that declares "static controls", and inherits from a code-behind class, and that code-behind class itself may or may not load dynamic controls.

Assuming this ASCX file were placed into the root of the application (denoted by "~/"), we could dynamically load it like this:

Control c = this.LoadControl("~/MyUserControl.ascx");

You might wonder why there's a difference in loading server controls and user controls. Why can't we create a user control just like a server control? This user control has a code-behind class "MyUserControl", so why can't we do this:

MyUserControl c = new MyUserControl(); // no!

I invented a new code sample CSS style just to illustrate this. Blue means good. Red means bad. You definitely can't create a user control like this.

In Part 1 remember we examined how the page parser converts your markup into a class that inherits from your code-behind class? And remember that the generated code creates the controls, adds them to the control tree, and assigns them to your code-behind's control variables that are declared as protected? User Controls work the same way.

By creating the code-behind class directly you have effectively by-passed this whole process. The result will be an instance of your control with no control tree. Your control definitely won't work, and will probably result in a NullReferenceException the first time your code actually tries to use any of the controls (because, well, they're null).

That is why the "LoadControl" method exists, and that is why the actual ASCX file must stick around. LoadControl retrieves and instantiates the type of the auto-generated code created from your markup.

The path given to LoadControl must be the virtual path to the ASCX file. Physical paths aren't allowed. The virtual path can take on one of three forms. Lets assume the application is running under virtual directory name "Truly" (in other words, http://myserver/truly represents the root of the application). The three forms are:

Control c;
c = this.LoadControl("MyUserControl.ascx"); // relative
c = this.LoadControl("./MyUserControl.ascx"); // relative
c = this.LoadControl("~/MyUserControl.ascx"); // app relative
c = this.LoadControl("/Truly/MyUserControl.ascx"); // root relative

Whatever the case, the virtual path must map inside the current application. No loading user controls that reside in other applications. So a root-relative path better map back into the current application ("Truly"), or you will get an error.

Relative paths are relative to the current page, or the current user control. That is important to know, because if you write a user control that loads another user control dynamically, you should know the difference between these:

Control c;
c = this.LoadControl("AnotherUserControl");
c = this.Page.LoadControl("AnotherUserControl");

The path is relative to the control you call the method on. The first line loads a control that exists in the same directory as the current user control. The second line loads a control that exists in the same directory as the current page, and the two don't have to be the same. Don't assume your user control is going to live in a particular location. It might be moved around. Really you have the same problem with Page.LoadControl, because you don't really know where in the directory structure the page loading your control will be.

It's usually best to keep user controls in one location, or at least segregated into multiple "silos" that logically separate them. That way you can always use a relative path like in the first line, yet you still have the freedom to move the controls around, just as long as you move them together. Pages that reference the controls will just have to know where they are, at least relatively.

You could also consider making the path to it configurable (via a property), use a web.config setting (less favorable), or use an app-relative path (ie, "~/UserControls/MyUserControl.ascx") and make the location known and unchangeable. 

 

Dynamically creating Parsed Controls

The ParseControl method is even more dynamic. It takes a string of markup and converts it into a control:

Control c;
c = this.ParseControl("Enter your name: <asp:TextBox id='txtFirstName' runat='server'/>");

The string can contain virtually anything you can put on a page (disclaimer: I say virtually, because while I don't know of a limitation, there may be one. If anyone reading this knows of a limitation let me know). All of the controls contained within will be put together into a single control, where it's control tree contains the parsed controls.

Parsed controls are interesting, but use them very carefully. Parsed controls carry a performance burden, because the string must be re-parsed every time. Normally the parsed result of pages or user controls is cached.

Here's a dramatic demonstration of the performance hit. Here we create the same user control over and over again (1 million times):

start = DateTime.Now;
for(int i = 0; i < 1000000; i++) {
    c = this.LoadControl("UserControls/MyUserControl.ascx");
}
end = DateTime.Now;
Response.Write((end - start).TotalSeconds);

And here we parse the content of the user control as a string over and over again:

start = DateTime.Now;
for(int i = 0; i < 1000000; i++) {
    c = this.ParseControl(
"Enter your name: <asp:TextBox id='txtFirstName' runat='server'/>");
}
end = DateTime.Now;
Response.Write((end - start).TotalSeconds);

And here are the results:

ParseControl took 253 seconds, loading the user control only took 19 seconds. That's a huge difference. Running ParseControl 1 million times in 253 seconds is still not a major performance issue (you're likely to have much more significant bottle necks in your application), but your mileage may vary, and you can't deny the performance results. Use it very sparingly. And definitely don't parse a string that came from a potentially malicious user! You would be enabling them to create any control that is available in the application, and they could use that to attack your site.

Any virtual paths you use inside the string follow the same rules as in the user control example. Relative paths are relative to the location of the control (or page) which you call ParseControl on. So there's that same subtle but important difference between the UserControl.ParseControl method and the Page.ParseControl method.

 

Dynamic Controls and Templates

There's another sneaky way that dynamic controls can be created. You have likely used templates without even realizing it. This topic is critical because if you understand how templates are used in databound controls like the Repeater, DataGrid, GridView, etc, then you will better recognize times where you may be approaching a problem with dynamic controls when you really don't have to. Later on there will be a section all about those situations.

Templates decrease the need for you to create dynamic controls manually because they let you declaratively define a "template" by which controls will be created dynamically for you. The Repeater control for example lets you define an ItemTemplate and optionally an AlternatingItemTemplate. Within the template you declaratively define controls:

<asp:Repeater ID="rpt1" runat="server">
    <ItemTemplate>
        Your name: <asp:TextBox ID="txtName" runat="server" Text="Enter your name" />
    </ItemTemplate>
</asp:Repeater>

The template contains a TextBox server control. Do you think the page that this repeater sits on has this TextBox, just like in our simple UserControl above? Do you think you can access this TextBox like this?

protected override void OnLoad(EventArgs e) {
    this.txtName.Text = "foo"; // no!
    base.OnLoad(e);
}

No no no. Remember -- red means bad. The TextBox you declared does not exist on the page, at all. There is no "build control" method generated for it, and it will not be assigned to any protected class variables you define (as described in Part 1).

Think about it for a minute. It doesn't even make sense for the TextBox to be accessible from the page like a regular statically declared control. The Repeater is going to be creating this TextBox once for every DataItem that is data bound to it. So there will be many of these on the page -- anywhere between 0 and N of them in fact. So if there were a page-level reference to it, which one would it point to? It doesn't make any sense.

When you declare a regular static controls, you are telling the framework "here is a control I'd like you to create and add to the page's control tree." When you define a template, you are saying "here is a control tree that I would like added to the page's control tree when the template is used." In the case of a repeater, the template is used once for every data item. But Templates don't have to be used that way (for example, the System.Web.UI.WebControls.Login control's LayoutTemplate property is a Template that isn't repeated). It's up to the control's implementation how the template is utilized.

Instead, the markup within the Template is parsed into an object that implements the System.Web.UI.ITemplate interface. Let's take a look at what code is generated for the static repeater in the example above:

private global::System.Web.UI.WebControls.Repeater @__BuildControlrpt1() {
    global::System.Web.UI.WebControls.Repeater @__ctrl;
 
    #line 12 "C:\projects\Truly\MyPage.aspx"
    @__ctrl = new global::System.Web.UI.WebControls.Repeater();
 
    #line default
    #line hidden
    this.rpt1 = @__ctrl;
 
    #line 12 "C:\projects\Truly\MyPage.aspx"
    @__ctrl.ItemTemplate = new System.Web.UI.CompiledTemplateBuilder(
        new System.Web.UI.BuildTemplateMethod(this.@__BuildControl__control4));
 
    #line default
    #line hidden
 
    #line 12 "C:\projects\Truly\MyPage.aspx"
    @__ctrl.ID = "rpt1";
 
    #line default
    #line hidden
    return @__ctrl;
}

Pay particular attention to the middle of this method. The repeater control has a property named ItemTemplate (which happens to accept objects of type ITemplate). The parser's generated code is creating a new CompiledTemplateBuilder and passing in a BuildTemplateMethod Delegate to it's constructor. The delegate is pointed at a method defined on this page, which happens to be this "build control" method:

private void @__BuildControl__control4(System.Web.UI.Control @__ctrl) {
    System.Web.UI.IParserAccessor @__parser = 
          ((System.Web.UI.IParserAccessor)(@__ctrl));
 
    #line 12 "C:\projects\Truly\MyPage.aspx"
    @__parser.AddParsedSubObject(new System.Web.UI.LiteralControl("\r\n Your name: "));
 
    #line default
    #line hidden
    global::System.Web.UI.WebControls.TextBox @__ctrl1;
 
    #line 12 "C:\projects\Truly\MyPage.aspx"
    @__ctrl1 = this.@__BuildControl__control5();
 
    #line default
    #line hidden
 
    #line 12 "C:\projects\Truly\MyPage.aspx"
    @__parser.AddParsedSubObject(@__ctrl1);
 
    #line default
    #line hidden
 
    #line 12 "C:\projects\Truly\MyPage.aspx"
    @__parser.AddParsedSubObject(new System.Web.UI.LiteralControl("    \r\n   "));
 
    #line default
    #line hidden
}

This "build control" auto-generated method is responsible for building the control tree that we declared within the repeater's ItemTemplate. As you can see, it creates a literal control, then calls the TextBox's build control method to create it, and then creates another literal control (the literal controls represent the non-server-control markup before and after the TextBox).

But this method isn't actually called from anywhere. It's simply the target of a Delegate, which was passed to the CompiledTemplateBuilder. So we haven't figured out yet exactly how the controls get into the control tree.

That's because it's up to the control containing the template to do something with the template. The page parser has done it's job. Now it's up to the repeater to do something about it. And of course, we already know the repeater "uses" it once for each data item. To see exactly how it uses it, lets look at the ITemplate interface:

public interface ITemplate {
    void InstantiateIn(Control container);
}

That's the entire interface. Just one method. The idea is that you have this control tree template, and when you need to instantiate it -- when you need to create an actual control tree based on the template (not unlike the relationship between a Class and an Instance of a Class) -- you call InstantiateIn(). The control tree is then created and added to the container you give it.

So we can summarize Repeater's use of templates as the following:

  1. Go to the next data item.
  2. Determine the appropriate item template for this data item (one of: ItemTemplate, AlternatingItemTemplate, HeaderTemplate, FooterTemplate).
  3. Create a container control (repeater uses a RepeaterItem).
  4. Call the selected template's InstantiateIn method, passing the container.
  5. Add the container to the repeater's control collection.
  6. Goto step 1 if there's any data items left.

In addition to this logic, the Repeater has an optional SeparatorTemplate, which it calls InstantiateIn() on between each data item.

Remember the "build control" method that the Delegate pointed to? Calling InstantiateIn() on the template is going to call that method. So the method is executed once for each data item, and thus that txtName TextBox we declared in the ItemTemplate markup is going to be created multiple times, in a dynamic manner.

So as you can see, Templates are powerful. And it also demonstrates that the framework itself is very much involved in the creation of dynamic controls. You can also create templates programmatically instead of statically by implementing the ITemplate interface yourself, but that's for a different blog entry.

All this and I still haven't gotten to the heart of the matter. The next part will examine how and when dynamic controls are added to the control tree. By "when" I mean when during the page lifecycle. That is so important because "when" can affect how the dynamic control participates in the lifecycle, and depending on the type of control and what features of it you are relying on, doing it at the wrong time will break it...

Part 1: Dynamic vs. Static
Part 2: Creating Dynamic Controls
Part 3: Adding Dynamic Controls to the Control Tree
Part 4: Because you don't know to render at design time

Just like ViewState, dynamic controls seem to be fodder for much debate, and a source of many confusing issues. This article will be the first of a multi-part series to detail just about everything you could ever want to know about how Dynamic Controls fit in with the framework.

ASP.NET has been out for half a decade. Maybe this article is a little late in the making. But with all of the new technologies coming down the pipe (e.g., Atlas), I thought it would be nice to get back to the basics. After all, not all of us have been in the game since the beginning :)

Actually, this article should probably be titled "TRULY Understanding ASP.NET". Because as you will see, dynamic controls really aren't that special. In learning how they fit in with the page life cycle, and how they are similar to static controls, you will gain an understanding of how ASP.NET works in general.

I decided to write this article as a direct consequence of the article I wrote on ViewState. Over the course of a few months I received dozens upon dozens of comments on that article from developers needing help. They assumed the trouble they were having was a ViewState related issue. But 75% of the time, it was actually a dynamic control issue!

Part of the problem is that so many developers think they need to create controls dynamically when there's often much more elegant or easier solutions. There's a tendency for developers to try and do "too much" themselves, and that leads to complex problems that seem to have no good solution. It's frustrating and discouraging. I've been there myself -- but I've learned that, almost always, when I find myself in that situation it's because I'm failing to see a bigger picture or I'm failing to approach the problem in an "asp.net way". I hope this article helps those who are experiencing the same thing.

I also decided to break this article up into multiple episodes, because the ViewState article is quite long, and I think this article is going to be even longer. Maybe if I give it to you in more bite size pieces it will be easier to read, no?

PART 1


 

Dynamic vs. Static: What's the difference?

Normally you "declare" controls on a form via markup and the ubiquitous runat="server" attribute. Like this:

<%@ Page Language="C#" AutoEventWireup="true"
CodeFile="MyPage.aspx.cs" Inherits="Infinity.Examples.MyPage" %>
<html xmlns="http://www.w3.org/1999/xhtml" >
<head runat="server">
    <title>TRULY Understanding Dynamic Controls</title>
</head>
<body>
    <form id="form1" runat="server">
        Your name: <asp:TextBox ID="txtName" runat="server" Text="Enter your name" />
    </form>
</body>
</html>

That TextBox is a "static" control. Static controls are declared in an xml-like syntax in ASPX or ASCX files. When ASP.NET parses that markup, it generates a class on the fly that does the dirty work for you -- one by one, the code creates the controls and adds them to the control tree. The TextBox declared above results in the following auto-generated code:

private global::System.Web.UI.WebControls.TextBox @__BuildControltxtName() {
    global::System.Web.UI.WebControls.TextBox @__ctrl;
 
    #line 11 "c:\projects\Truly\MyPage.aspx"
    @__ctrl = new global::System.Web.UI.WebControls.TextBox();
 
    #line default
    #line hidden
    this.txtName = @__ctrl;
    @__ctrl.ApplyStyleSheetSkin(this);
 
    #line 11 "c:\projects\Truly\MyPage.aspx"
    @__ctrl.ID = "txtName";
 
    #line default
    #line hidden
 
    #line 11 "c:\projects\Truly\MyPage.aspx"
    @__ctrl.Text = "Enter your name";
 
    #line default
    #line hidden
    return @__ctrl;
}

The code may look scary, but it's just a really careful way of creating a TextBox, setting the ID and Text properties, and returning it. It has to be careful because it's possible that user code define things that make change the intended purpose. The #line pragmas are so the compiler can alert you to any compile errors and still give you the correct line number in the declared markup, rather than the line number of the generated code (say thank you to the compiler next time that happens -- its a great feature).

There will be one of these "build control" methods for each control you statically declare on the page. If a control exists inside another control, then the parent control's "build control" method will add them to it's own control collection. In this example, the form control contains three child controls:

private global::System.Web.UI.HtmlControls.HtmlForm @__BuildControlform1() {
    global::System.Web.UI.HtmlControls.HtmlForm @__ctrl;
 
    #line 10 "c:\projects\Truly\MyPage.aspx"
    @__ctrl = new global::System.Web.UI.HtmlControls.HtmlForm();
 
    #line default
    #line hidden
    this.form1 = @__ctrl;
 
    #line 10 "c:\projects\Truly\MyPage.aspx"
    @__ctrl.ID = "form1";
 
    #line default
    #line hidden
    System.Web.UI.IParserAccessor @__parser = ((System.Web.UI.IParserAccessor)(@__ctrl));
 
    #line 10 "c:\projects\Truly\MyPage.aspx"
    @__parser.AddParsedSubObject(new System.Web.UI.LiteralControl("\r\n  Your name: "));
 
    #line default
    #line hidden
    global::System.Web.UI.WebControls.TextBox @__ctrl1;
 
    #line 10 "c:\projects\Truly\MyPage.aspx"
    @__ctrl1 = this.@__BuildControltxtName();
 
    #line default
    #line hidden
 
    #line 10 "c:\projects\Truly\MyPage.aspx"
    @__parser.AddParsedSubObject(@__ctrl1);
 
    #line default
    #line hidden
 
    #line 10 "c:\projects\Truly\MyPage.aspx"
    @__parser.AddParsedSubObject(new System.Web.UI.LiteralControl("\r\n    "));
 
    #line default
    #line hidden
    return @__ctrl;
}

The child controls are:

  1. LiteralControl containing the straight-up text "Your name:"
  2. The txtName TextBox
  3. Another LiteralControl containing the whitespace before the end of the form tag.

Notice to create the TextBox control, this method calls that control's "build method" that we just looked at:

global::System.Web.UI.WebControls.TextBox @__ctrl1;
#line 10 "c:\projects\Truly\MyPage.aspx"
@__ctrl1 = this.@__BuildControltxtName.@__BuildControltxtName();

And notice this strange method it calls, passing the TextBox:

@__parser.AddParsedSubObject(@__ctrl1);

Because of the fact the "object" being added is a control, all that method does is call Controls.Add(@__ctrl1), thus adding the TextBox to the control tree (I say "because of the fact it is a control", because non-controls can be added as well, but how they are handled is unique to each control type).

This isn't shown in this example, but haven't you ever wondered why you declare control variables as protected? Why not private? The class that is auto-generated inherits from your code-behind (that's why you need an "inherits" attribute in the page directive). It's up to this code to create the declared controls and assign them to the variables you declare (if it exists) so that you can easily access them. Since it is derived from your code-behind class, if the variable were private, it couldn't do that!

So that is how the framework adds declared controls into the control tree. A dynamic control in the traditional sense is one that you as the page or control developer create at some point during the page lifecycle, and add to the static control tree yourself. Something like this:

protected override void OnInit(EventArgs e) {
    TextBox txtName = new TextBox();
    txtName.ID = "txtName";
    txtName.Text = "Enter your name";
    this.form1.Controls.Add(txtName);
 
    base.OnInit(e);
}

This code is not nearly as scary looking as the auto-generated code from the page parser. However, it does the same thing!

So you see, this whole silly ASPX-markup-runat-server-thing exists solely to make it easier on you to create a control tree! That's all it's there for! If it didn't exist, you could still achieve all the same functionality by creating the controls and setting their properties yourself, just like the auto-generated code does! When I first realized this, it was a eureka moment for me. So if you don't feel the same way, either you don't understand what I'm saying, or I'm crazy when compared to you. Maybe it's a little of both :)

This really blurs the line between static and dynamic controls, as it should. What you think of as a static control, is really just a control that is being dynamically created by the framework instead of by you. But neither the control or the framework really care who creates it -- they can participate in the page lifecycle one and the same, and both the control and the page it sits on don't even know the difference.

But wait!

If you have ever dealt with dynamic controls before, you know that they really are different. It's true that your experience with them may differ from that of static controls, but it's important to understand exactly why there's a difference. The difference has to do with when the control enters the control tree. When the framework adds controls, it does it extremely early in the event sequence. The only thing that happens first is the control or page constructor. That's why in OnInit or OnPreInit, the controls already exist and are ready for use (well, master pages can 'mess' with that process, but this is still a true statement).

But when you dynamically create a control, you don't have the ability to do it as early as the framework does. And that has consequences...

Stay tuned. :)

Most people don't even realize you can just say "throw;" within a catch block to re-throw the original exception. But my good friend Matt has pointed out an important difference that goes all too unnoticed these days. So important, it could mean the difference in a debug session between "ah ha!" and "what the?"

Anyway, if you don't know what the difference is between this:

try {
    // do something bad
}
catch(Exception ex) {
    // log the error somewhere
    LogException(ex);
    // rethrow it
    throw ex;
}

and this:

try {
    // do something bad
}
catch(Exception ex) {
    // log the error somewhere
    LogException(ex);
    // rethrow it
    throw;
}

Then you should definitely read about it here. Now that being said, there's always the debate about whether one should ever be catching the general Exception (I personally believe there are definitely times where it is appropriate), but whatever side of that fence you are on, it's still important to know how to properly rethrow an exception.

A very exciting new feature in ASP.NET 2.0 is Expression Builders. Expression builders allow for some pretty interesting interaction with the ASP.NET compilation model.

For example, new to ASP.NET 2.0 is the ability to reference appSettings declaratively. Lets say you wanted the text of a button to be based on a value in the appSettings section of your web.config. Piece of cake:

<asp:Button id="cmdSubmit" runat="server" Text="<%$ appSettings: ButtonText %>" />

This is made possible by the built in AppSettingsExpressionBuilder. Lets say you wanted to display a localizable string, which is stored as a resource. In ASP.NET 1.1 it was sort of a pain. No longer:

<%$ resources: ResourceKey %>

That's the ResourceExpressionBuilder hard at work. Nice!

There's also a ConnectionStringExpressionBuilder so you can refer to Connection Strings defined in the new connection strings section in the web.config. That is extremely helpful when working with the new declarative data controls:

<asp:SqlDataSource id="data1" runat="server"
    ConnectionString="<%$ ConnectionStrings: MyConnectionString %>"/>

Pretty useful.

Now lets take it one step further. Have you ever seen this exception?


The dreaded "Server tags cannot contain constructs" exception.

That's because you probably tried to do something like this:

<asp:Label id="lbl1" runat="server" Text=<%= CurrentUserName %> />

Perhaps you tried to fix it by putting it in quotes:

<asp:Label id="Label1" runat="server" Text="<%= CurrentUserName %>" />

... Only to be thwarted once again, as the literal text, including the <%= %> construct, ended up in the page:


Putting <%= %> in quotes doesn't help much.

If all you want to do is show the result of this code as a string, you could quite simply just get rid of the label:

<%= CurrentUserName %>

That would work. But perhaps you need the label server control for some other reason, or perhaps you need to set a string property of some other type of server control in this way.

The problem is you have incorrectly (although intuitively) tried to assign the property of a server control using the <%= %> construct. Unfortunately that is simply not supported by ASP.NET, 1.1 nor 2.0. If you ask around about your problem, someone may tell you that you will have to convert to using the <%# %> databinding construct instead. That is advice that I have given myself. But it requires that you are calling DataBind() on the control, AND, it will cause you to BLOAT VIEWSTATE... and you KNOW how much I hate bloating viewstate!

You can just go ahead and assign the value in your code-behind, say.. in the OnInit method.. but that too will bloat viewstate. I'm afraid there's no super simple solution unless you don't mind disabling viewstate on that control. That may work in some scenarios, but sometimes you really need the ViewState enabled! What's a web developer to do?

Let me back up a little and give a more concrete example. You want the text value of a CheckBox to be the current date and time. For whatever reason, you can't disable viewstate on the CheckBox (say, because you need the CheckChanged event, which doesn't work without viewstate). How on Earth are you going to get the current date and time into the Text property on the CheckBox, WITHOUT BLOATING VIEWSTATE? By "Bloating ViewState", I mean causing data to become serialized in the __VIEWSTATE hidden form field when it isn't necessary to begin with. There's no reason to put the current Date and Time into serialized viewstate, is there? It's going to be reassigned on the next request. You'd just be making ASP.NET serialize it, then making the user's browser pull the serialized string down the pipe, then making them push it back up the pipe to the server on a postback, then making ASP.NET deserialize the value -- ONLY FOR IT TO BE REASSIGNED? How rude! How wasteful and inefficient. For a single control its not a big deal, what's a few bytes? But that is not a path you want to start down my friend... with that kind of mantra, your web forms will quickly grow a viewstate tumor the size of a Borg cube! Ideally, you want the functional equivalence of this:

<asp:CheckBox id="chk1" runat="server" Text="<%= DateTime.Now %>"/>

That is ideal, because the ViewState StateBag is not tracking changes when ASP.NET assigns declared attributes, so our beloved ViewState remains optimized. And, we didn't even have to disable ViewState. AND we can do it declaratively! Woohoo! Right. Well, it won't work.

The CodeExpressionBuilder comes to the rescue! Another great thing about ExpressionBuilders is that you can roll your own. So, I created a CodeExpressionBuilder, one that allows you to use raw code to assign values to control properties. Using the CodeExpressionBuilder you can do this:

<asp:CheckBox id="chk1" runat="server" Text="<%$ Code: DateTime.Now %>"/>

Now that is nice. And to think the darn thing is only a few lines of code! The trick is to use the CodeDom's CodeSnippetExpression to convert the given string into a CodeExpression. Here's the entire class:

[ExpressionPrefix("Code")]
public class CodeExpressionBuilder : ExpressionBuilder {
    public override CodeExpression GetCodeExpression(BoundPropertyEntry entry, 
       object parsedData, ExpressionBuilderContext context) {
        return new CodeSnippetExpression(entry.Expression);
    }
}

To use it, or any custom ExpressionBuilder for that matter, you must register it in the web.config expressionBuilders section. Now... how you do this part sort of depends on how your project is setup. If you have a standard ASP.NET Web Site project, then you will be defining the CodeExpressionBuilder class in the app_code directory, and the "type" will just be "CodeExpressionBuilder". However, if you are creating a Web Application Project (read about it here), then the CodeExpressionBuilder is just another class in your project, with its own namespace. For that you will need to define the whole type string (or, if you define it in a reusable library, you'll need the fully qualified type and assembly name). In my case that is "Infinity.Web.Compilation.CodeExpressionBuilder". Like this:

<compilation debug="true">
  <expressionBuilders>
    <add expressionPrefix="Code" type="Infinity.Web.Compilation.CodeExpressionBuilder"/>
  </expressionBuilders>
</compilation>

And to see it in action:

<asp:CheckBox id="chk1" runat="server" Text="<%$ Code: DateTime.Now %>" />

ExpressionBuilders are truly a thing of beauty! Use any expression you want!

Also try:

<%$ Code: DateTime.Now.AddDays(1) %>
<%$ Code: "Hello World, " + CurrentUserName %>
<%$ Code: CurrentUserName.ToUpper() %>
<%$ Code: "Page compiled as: " + this.GetType().AssemblyQualifiedName %>

Just be careful what combination of quotes you use. If you have literal strings in your code expression like in two of the above examples, you will need to use single quotes (') to surround the entire <%$ %> expression if it is within a server control declaration. If you don't you will get the "Server tag is not well formed" error.

In the beginning I said ExpressionBuilders were an interesting way to plug into the ASP.NET compilation model. Well this really illustrates that... put a break point in the expression builder, and debug the page. You will hit the break point once, and only once, even after you refresh the page several times. The Date and Time will continue to update, but the expression builder breakpoint will only activate the first time you hit the page. The reason is because the ExpressionBuilder is used when ASP.NET compiles the page. Once the page is compiled, that's it. That's the reason why ExpressionBuilder returns a CodeExpression, and not an actual object. In essence, the builder tells ASP.NET what code it needs to run to get the value, instead of giving it the actual value. It's the old adage, teach a man to fish, and he eats forever. How geeky is that? Too cool.

PS: The one thing this ExpressionBuilder doesn't do... it won't work in "No-Compile" pages. Seems like a reasonable limitation.

Questions? Comments? Random odd facts?

HAPPY (dynamic) CODING!

UPDATE 09/05/2006: Thanks to the tip from Kelly, made the code even shorter. Also fixed a typo in the web.config snippet. The "Type" attribute should be lower case, "type".

Visual Studio is without a doubt a powerful tool. With every iteration, it continues to improve upon itself. But as you happily hack away at all your applications, you are blistfully unaware of it's evil dark side that has been there since the beginning. It's true.

There are those of us who embrace the dark side. But we are out numbered...

You see, the dark side isn't how it comes by default. No... it comes all happy and bright and cheery by default, and like good little jedi programmers you accept those defaults. But the dark side is there, hidden away deep within the environment settings, reaching out and corrupting those programmers who are corruptible. Why some are corruptible and some are not is a mystery that may never be solved, but each and every programmer must give pause and consider the benefits it provides.

The Dark Side

I know what you may be thinking. It's hard to read. Are you so sure about that? Lets compare it with the happy cheery default color scheme:

The Status Quo

The key difference of course is the pervasive black background color. This is the environment I and many others work in every day. For me, it started way back when I used a Borland C++ IDE that had preset environment color schemes. One was called Twilight, and it looked similar to this. Most people who see it for the first time are offended by it... but if you think about it, it really makes sense. It brings balance to the force.

The default scheme sports a bright white background color with dark text over it. But monitors these days are brighter than ever. You're presumably a programmer, so you've no doubt had those late but productive coding nights, nights that are lit by only the glow of your monitor. The glow is bright enough to light up the room and cast shadows. Not unlike... a light bulb.

A Light Bulb

So there you are, staring straight into a strong light source, looking for the few pixels on it which are not illuminated. Can you read the wattage and manufacturer letters on the head a light bulb while it's turned on? Ahhh... but what if the bulb were black, and only the letters on it were illuminated?

A Light Bulb 

This bulb has no markings, but you'd bet they'd show up nice and bright and easy to read in the right image. Another benefit someone pointed out to me once -- if you're on a laptop, it saves your battery life! Horray for an extra 20 minutes of mobile coding!

It seems to me the only reason a black-on-white background is so standard is because the GUI was invented to be an analogy to pen and paper. Paper is white. Your screen doesn't have to be. Don't conform to the status quo! Plus, it just looks really cool... I think.

Want to join the dark side? Download the settings here..

Visual Studio 2005
Visual Studio 2008

Just import them via the Tools->Import and Export Settings menu. Don't worry. If it doesn't suit you, just revert :)

As for VS2003, to enable exporting/importing styles in VS2003 you need an add-in. There happens to be one called VSStyler, which you can download here:

VSStyler Add-In for VS2003

Once you have the add-in, download the dark side style export here:

The Dark Side for VS2003

UPDATE 03/28/2008: Added new settings for Visual Studio 2008. I've improved on the scheme a little, and it now uses the Consolas font, too.

Happy (dark) coding!

ViewState is a very misunderstood animal. I would like to help put an end to the madness by attempting to explain exactly how the ViewState mechanism works, from beginning to end, and from many different use cases, such as declared controls vs. dynamic controls.

There are a lot of great articles out there that try to dispel the myths about ViewState. You might say this is like beating a dead horse (where ViewState is the horse, and the internet is the assailant). But this horse isn't dead, let me tell you. No, he's very much alive and he's stampeding through your living room. We need to beat him down once again. Don't worry, no horses were harmed during the authoring of this article.

It's not that there's no good information out there about ViewState, it's just all of them seem to be lacking something, and that is contributing to the community's overall confusion about ViewState. For example, one of the key features that is important to understand about ViewState is how it tracks dirtiness. Yet, here is a very good, in-depth article on ViewState that doesn't even mention it! Then there's this W3Schools article on ViewState that seems to indicate that posted form values are maintained via ViewState, but that's not true. (Don't believe me? Disable ViewState on that textbox in their example and run it again). And it's the #1 Google Search Result for "ASP.NET ViewState". Here is ASP.NET Documentation on MSDN that describes how Controls maintain state across postbacks. The documentation isn't wrong per say, but it makes a statement that isn't entirely correct:

"If a control uses ViewState for property data instead of a private field, that property automatically will be persisted across round trips to the client."

That seems to imply that anything you shove into the ViewState StateBag will be round-tripped in the client's browser. NOT TRUE! So it's really no wonder there is so much confusion on ViewState. There is no where I've found on the internet that has a 100% complete and accurate explanation of how it works! The best article I have ever found is this one by Scott Mitchell. That one should be required reading. However, it does not explain the relationship of controls and their child controls when it comes to initialization and ViewState Tracking, and it is this point alone that causes a bulk of the mishandlings of ViewState, at least in the experiences I've had.

So the point of this article will be to first give a complete understanding of how ViewState basically functions, from beginning to end, hopefully filling in the holes that many other articles have. After a complete explanation of the entire ViewState process, I will go into some examples of how developers typically misuse ViewState, usually without even realizing it, and how to fix it. I should also preface this with the fact that I wrote this article with ASP.NET 1.x in mind. However, there are very few differences in the ViewState mechanism in ASP.NET 2.0. For one, ControlState is a new type of ViewState in ASP.NET 2.0, but it treated exactly like ViewState, so we can safely ignore it for the purposes of this article.

First let me explain why I think understanding ViewState to it's core is so important:

    MISUNDERSTANDING OF VIEWSTATE WILL LEAD TO...
  1. Leaking sensitive data
  2. ViewState Attacks - aka the Jedi Mind Trick -- *waves hand* that plasma tv is for sale for $1.00
  3. Poor performance - even to the point of NO PERFORMANCE
  4. Poor scalability - how many users can you handle if each is posting 50k of data every request?
  5. Overall poor design
  6. Headache, nausea, dizziness, and irreversible frilling of the eyebrows.
If you develop an ASP.NET Application and you don't take ViewState seriously, this could happen to you:
ViewState Madness!!! Drop your red bull and surrender your cpu cycles. You will be frustrated. Performance is futile!
The ViewState form field. ViewState will add your web app's distinctiveness to it's own. Performance is futile.
I could go on but that is the gist of it. Now lets move on by starting back from the beginning:
    WHAT DOES VIEWSTATE DO?
    This is a list of ViewState's main jobs. Each of these jobs serves a very distinct purpose. Next we'll learn exactly how it fulfills those jobs.
  1. Stores values per control by key name, like a Hashtable
  2. Tracks changes to a ViewState value's initial state
  3. Serializes and Deserializes saved data into a hidden form field on the client
  4. Automatically restores ViewState data on postbacks
Even more important than understanding what it does, is understanding what it does NOT do:
    WHAT DOESN'T VIEWSTATE DO?
  1. Automatically retain state of class variables (private, protected, or public)
  2. Remember any state information across page loads (only postbacks) (that is unless you customize how the data is persisted)
  3. Remove the need to repopulate data on every request
  4. ViewState is not responsible for the population of values that are posted such as by TextBox controls (although it does play an important role)
  5. Make you coffee
While ViewState does have one overall purpose in the ASP.NET Framework, it's four main roles in the page lifecycle are quite distinct from each other. Logically, we can separate them and try to understand them individually. It is often the mishmash of information on ViewState that confuses people. Hopefully this breaks it down into more bite size nuggets. Mmmm... ViewState Nuggets.
ViewState Nuggets

1. VIEWSTATE STORES VALUES
If you've ever used a hashtable, then you've got it. There's no rocket science here. ViewState has an indexer on it that accepts a string as the key and any object as the value. For example:

ViewState["Key1"] = 123.45M; // store a decimal value
ViewState["Key2"] = "abc"; // store a string
ViewState["Key3"] = DateTime.Now; // store a DateTime

Actually, "ViewState" is just a name. ViewState is a protected property defined on the System.Web.UI.Control class, from which all server controls, user controls, and pages, derive from. The type of the property is System.Web.UI.StateBag. Strictly speaking, the StateBag class has nothing to do with ASP.NET. It happens to be defined in the System.Web assembly, but other than it's dependency on the State Formatter, also defined in System.Web.UI, there's no reason why the StateBag class couldn't live along side ArrayList in the System.Collections namespace. In practice, Server Controls utilize ViewState as the backing store for most, if not all their properties. This is true of almost all Microsoft's built in controls (ie, label, textbox, button). This is important! You must understand this about controls you are using. Read that sentance again. I mean it... here it is a 3rd time: SERVER CONTROLS UTILIZE VIEWSTATE AS THE BACKING STORE FOR MOST, IF NOT ALL THEIR PROPERTIES. Depending on your background, when you think of a traditional property, you might imagine something like this:

public string Text {
    get { return _text; }
    set { _text = value; }
}

What is important to know here is that this is NOT what most properties on ASP.NET controls look like. Instead, they use the ViewState StateBag, not a private instance variable, as their backing store:

public string Text {
    get { return (string)ViewState["Text"]; }
    set { ViewState["Text"] = value; }
}

And I can't stress it enough -- this is true of almost ALL PROPERTIES, even STYLES (actually, Styles do it by implementing IStateManager, but essentially they do it the same way). When writing your own controls it would usually be a good idea to follow this pattern, but thought should first be put into what should and shouldn't be allowed to be dynamically changed on postbacks. But I digress -- that's a different subject. It is also important to understand how DEFAULT VALUES are implemented using this technique. When you think of a property that has a default value, in the traditional sense, you might imagine something like the following:

public class MyClass {
    private string _text = "Default Value!";
 
    public string Text {
        get { return _text; }
        set { _text = value; }
    }
}

The default value is the default because it is what is returned by the property if no one ever sets it. How can we accomplish this when ViewState is being used as the private backing? Like this:

public string Text {
    get {
        return ViewState["Text"] == null ?
             "Default Value!" :
              (string)ViewState["Text"];
    }
    set { ViewState["Text"] = value; }
}

Like a hashtable, the StateBag will return null as the value behind a key if it simply doesn't contain an entry with that key. So if the value is null, it has not been set, so return the default value, otherwise return whatever the value is. For you die-hards out there -- you may have detected a difference in these two implementations. In the case of ViewState backing, setting the property to NULL will result in resetting the property back to it's default value. With a "regular" property, setting it to null means it will simply be null. Well, that is just one reason why ASP.NET always tends to use String.Empty ("") instead of null. It's also not very important to the built in controls because basically all of their properties that can be null already are null by default. All I can say is keep this in mind if you write your own controls. And finally, as a footnote really, while this property-backing usage of the ViewState StateBag is how the StateBag is typically used, it isn't limited to just that. As a control or page, you can access you're own ViewState StateBag at any time for any reason, not just in a property. It is sometimes useful to do so in order to remember certain pieces of data across postbacks, but that too is another subject.

2. VIEWSTATE TRACKS CHANGES
Have you ever set a property on a control and then somehow felt... dirty? I sure have. In fact, after a twelve-hour day of setting properties in the office, I become so filthy my wife refuses to kiss me unless I'm holding flowers to mask the stench. I swear! Ok so setting properties doesn't really make you dirty. But it does make the entry in the StateBag dirty! The StateBag isn't just a dumb collection of keys and values like a Hashtable (please don't tell Hashtable I said that, he's scarey). In addition to storing values by key name, the StateBag has a TRACKING ability. Tracking is either on, or off. Tracking can be turned on by calling TrackViewState(), but once on, it cannot be turned off. When tracking is ON, and ONLY when tracking is ON, any changes to any of the StateBag's values will cause that item to be marked as "Dirty". StateBag even has a method you can use to detect if an item is dirty, aptly named IsItemDirty(string key). You can also manually cause an item to be considered dirty by calling SetItemDirty(string key). To illustrate, lets assume we have a StateBag that is not currently tracking:

stateBag.IsItemDirty("key"); // returns false
stateBag["key"] = "abc";
stateBag.IsItemDirty("key"); // still returns false
 
stateBag["key"] = "def";
stateBag.IsItemDirty("key"); // STILL returns false
 
stateBag.TrackViewState();
stateBag.IsItemDirty("key"); // yup still returns false
 
stateBag["key"] = "ghi";
stateBag.IsItemDirty("key"); // TRUE!
 
stateBag.SetItemDirty("key", false);
stateBag.IsItemDirty("key"); // FALSE!

Basically, tracking allows the StateBag to keep track of which of it's values have been changed since TrackViewState() has been called. Values that are assigned before tracking is enabled are not tracked (StateBag turns a blind eye). It is important to know that any assignment will mark the item as dirty -- even if the value given matches the value it already has!

stateBag["key"] = "abc";
stateBag.IsItemDirty("key"); // returns false
stateBag.TrackViewState();
stateBag["key"] = "abc";
stateBag.IsItemDirty("key"); // returns true

ViewState could have been written to compare the new and old values before deciding if the item should be dirty. But recall that ViewState allows any object to be the value, so you aren't talking about a simple string comparison, and the object doesn't have to implement IComparable so you're not talking about a simple CompareTo either. Alas, because serialization and deserialization will be occuring, an instance you put into ViewState won't be the same instance any longer after a postback. That kind of comparison is not important for ViewState to do it's job, so it doesn't. So that's tracking in a nutshell.

But you might wonder why StateBag would need this ability in the first place. Why on earth would anyone need to know only changes since TrackViewState() is called? Why wouldn't they just utilize the entire collection of items? This one point seems to be at the core of all the confusion on ViewState. I have interviewed many professionals, sometimes with years and years of ASP.NET experience logged in their resumes, who have failed miserably to prove to me that they understand this point. Actually, I have never interviewed a single candidate who has! First, to truly understand why Tracking is needed, you will need to understand a little bit about how ASP.NET sets up declarative controls. Declarative controls are controls that are defined in your ASPX or ASCX form. Here:

<asp:Label id="lbl1" runat="server" Text="Hello World" />

I do declare that this label is declared on your form. The next thing we need to make sure you understand is ASP.NET's ability to wire up declared attributes to control properties. When ASP.NET parses the form, and finds a tag with runat=server, it creates an instance of the specified control. The variable name it assigns the instance to is based on the ID you assigned it (by the way, many don't realize that you don't have to give a control an ID at all, ASP.NET will use an automatically generated ID. Not specifying an ID has advantages, but that is a different subject). But that's not all it does. The control's tag may contain a bunch of attributes on it. In our label example up above, we have a "Text" attribute, and it's value is "Hello World". Using reflection, ASP.NET is able to detect whether the control has property by that name, and if so, sets its value to the declared value. Obviously the attribute is declared as a string (hey, its stored in a text file after all), so if the property it maps to isn't of type string, it must figure out how to convert the given string into the correct type, before calling the property setter. How it does that my friend is also an entirely different topic (it involves TypeConverters and static Parse methods). Suffice it to say it figures it out, and calls the property setter with the converted value.

Recall that all-important statement from the first role of the StateBag. Here it is again: Server Controls utilize ViewState as the backing store for most, if not all their properties. That means when you declare an attribute on a server control, that value is usually ultimately stored as an entry in that control's ViewState StateBag. Now recall how tracking works. Remember that if the StateBag is "tracking", then setting a value to it will mark that item as dirty. If it isn't tracking, it won't be marked dirty. So the question is -- when ASP.NET calls the SET on the PROPERTY that corresponds to the ATTRIBUTE that is DECLARED on the control, is the StateBag TRACKING or isn't it? The answer is no it is not tracking, because tracking doesn't begin until someone calls TrackViewState() on the StateBag, and ASP.NET does that during the OnInit phase of the page/control lifecycle. This little trick ASP.NET uses to populate properties allows it to easily detect the difference between a declaratively set value and dynamically set value. If you don't yet realize why that is important, please keep reading.

3. SERIALIZATION AND DESERIALIZATION
Aside from how ASP.NET creates declarative controls, the first two capabilities of ViewState we've discussed so far have been strictly related to the StateBag class (how it's similar to a hashtable, and how it tracks dirty values). Here is where things get bigger. Now we will have to start talking about how ASP.NET uses the ViewState StateBag's features to make the (black) magic of ViewState happen.

If you've ever done a "View Source" on an ASP.NET page, you've no doubt encountered the serialization of ViewState. You probably already knew that ViewState is stored in a hidden form field aptly named _ViewState as a base64 encoded string, because when anyone explains how ViewState works, that's usually the first thing they mention.

A brief aside -- before we understand how ASP.NET comes up with this single encoded string, we must understand the hierarchy of controls on the page. Many developers with years of experience still don't realize that a page consists of a tree of controls, because all they work on are ASPX pages, and all they need to worry about are controls that are directly declared on those pages... but controls can contain child controls, which can contain their own child controls, etc. This forms a tree of controls, where the ASPX page itself is the root of that tree. The 2nd level is all the controls declared at the top level in the ASPX page (usually that consists of just 3 controls -- a literal control to represent the content before the form tag, a HtmlForm control to represent the form and all its child controls, and another literal control to represent all the content after the close form tag). On the 3rd level are all the controls contained within those controls (ie, controls that are declared within the form tag), and so on and so forth. Each one of the controls in the tree has it's very own ViewState -- it's very own instance of a StateBag. There's a protected method defined on the System.Web.UI.Control class called SaveViewState. It returns type 'object'. The implementation for Control.SaveViewState is to s