Scott Forsyth's Blog

Postings on IIS, ASP.NET, SQL Server, Webfarms and general system admin.

ASP.NET v2.0 - AppDomain recycles, more common than before

I stumbled across a very interesting find yesterday morning and Todd Carter confirmed it by sending a link to his blog entry.  (it's a good read)

First let me cover briefly what an AppDomain is. ASP.NET (.NET Winforms too but I won't cover that here) starts and runs within a sandbox called an AppDomain. Each Application on a server will run its own AppDomain which allows separation between different parts of the site for better security isolation and availability. If a folder is marked as an application, it will live within a different AppDomain than the root of the site. This information is not easily seen in IIS6 and earlier versions of IIS, but AppDomains are there just the same.

What this means though is that if the AppDomain is restarted, then there is a performance hit while the AppDomain is started up and pages are recompiled on their first hit. Furthermore, session state that is running In-Process is lost. In-Process, or "InProc" session state is the default setting.

In ASP.NET v1.x, there is an AppDomain restart when global.asax, web.config or anything related to the /bin folder is 'touched'. By 'touched', I mean renaming, creating, or updating the file in any way. And of course an IIS6 application pool recycle or a restart of IIS would cause the old AppDomain to shut down and a new one to start. This all made sense because those were application files and any change to them would make significant changes to the running AppDomain.

While troubleshooting an unrelated issue on a v2.0 site yesterday, I noticed that there was an AppDomain restart simply by creating a non-application folder. Through further testing I found that adding folders, deleting folders and renaming folders caused the AppDomain recycle. Furthermore, if many dozen files were added in short succession to each other, that would also case the same restart. Of course the app_{folders} will cause a restart, and that makes sense, but completely non-application related folders also cause this same behavior.

It turns out that this was a purposeful change with v2.0 and was done so that stale content doesn't remain in place after a significant change. I guess there were enough situations where that happened that they changed the situations that caused an AppDomain recycle to be far more aggressive than v1.x of the framework.

What does this mean? Unfortunately quite a bit. People that used to make quick updates to their websites thinking that it wouldn't affect performance of unrelated parts of the site now need to consider that any changes to folders will cause the entire AppDomain to be restarted. Also a large number of changes to files within the site will do the same. (I'm not sure at this time the threshold that triggers a restart based on file changes). This will immediately kill all InProc session state and cause a performance hit on the site.

I'm normally not one to complain about .NET, there are so many great things to say about it. But in this situation, I think this is a step backward. At this time, there is no way to disable this aggressive response to file and folder changes. The linkd.exe solution that Todd gave in the blog entry is a solution, but it's not an easy one and isn't reasonable in many situations.

I don't have a good solution for this, but at least I am now aware of the affects of various file and folders changes and can watch out for that when making site updates or troubleshooting sites.

Posted: Feb 21 2006, 09:20 AM by OWScott | with 30 comment(s)
Filed under:

Comments

Ted N. said:

Besides losing the Inproc Session state on a recycle, Do you lose all of your cached items as well?
# February 21, 2006 1:04 PM

Scott Forsyth said:

Yep, the cache is lost as well.
# February 21, 2006 5:16 PM

saurabh nandu said:

This is a very intresting observation. in fact a very trouble some one too.. many times there are temp files like XML/XSLT that get generated and deleted on the fly for applications (like file system based cache's etc.) Now if these were to trigger an app domain restart that would really suck!
# February 22, 2006 2:12 AM

Chad Humphries said:

This seems like it would cripples systems that allow users to manage files through the site, or systems that allow uploads/directory creation of any form.

Does this happen only in the new asp.net model, or does it also occur in the recently released web project model as well?

# February 22, 2006 10:11 AM

Scott Forsyth said:

Yes, it can have a crippling affect on the site. In fact that is how I discovered it. Changes to a small number of files won't hurt anything, but if there is a large number of file changes, I believe 15 is the magic number, in a very short time, it will cause this. "Any" folder changes will cause this.

I'm still working with Todd on a particular case and he just let me know that the App_Data folder is excluded from the FCN (file change notification) checking, so that is a safe place to put things that change often. I tested and confirmed this is the case.

This is a function of the v2.0 framework so as far as I know the web project model doesn't change this at all. It will happen there as well.
# February 22, 2006 10:30 AM

Chad Humphries said:

Hopefully this is documented somewhere or another. Otherwise, you have to love magic settings and values like this. Anyway, who would ever create folders and files while the app is running? :) That's crazy talk, I'm sure only 65-85% of app devs do it.
# February 22, 2006 11:53 AM

OldManChild said:

I developed an application using Version 2.0.50215 of the Framework.Using a combination of command line aspnet_regiis.exe -ua and aspnet_regiis.exe -i I uninstalled all version of the framework and reinstalled version v2.0.50727. I keep getting the Server Application Unavailable message in the browser.
In the event log I get a message with Event ID:1088 and a message of "Failed to execute request because the App-Domain could not be created. Error: 0x80004005 Unspecified error ". What could be the cause? Been on it for 3 days
# February 23, 2006 11:40 AM

Scott Forsyth said:

Hi OldManChild,

Anything can happen if your machine had the beta bits on it. I've seen some cleanup tools around though although I don't have any bookmarked, but you might want to search for any type of beta .net v2.0 uninstall tools and see if that helps.

Often Server Application Unavailable is from disk permission issues, or a custom app pool user not being added to the IIS_WPG group.

A great tool to use for troubleshooting disk level issues is filemon from www.sysinternals.com. It's easy to pick up and free. That will let you know if you're fighting file/folder permissions.
# February 24, 2006 11:07 AM

OldManChild said:

Thanks Scott,
I hate giving up but I had to.After countless permutations and combinations of varying installs and uninstalls,I just reinstalled Windows Server 2003 and progressively installed all my programs.Glad to say I am back at Version 2.0.50215 and Visual Studio 2005 Beta. Planning to buy the release version and install that.
Latest problem is getting constructing a TreeView dynamically and programmatically from an HttpHandler.Keep getting an "Object Reference not set to an instance of an object" error. But I guess this is not the forum for that.......
# February 27, 2006 12:13 PM

Scott Forsyth said:

OldManChild, sometimes it comes to a reinstall after working with Beta builds. If you've exhausted all your options and it still doesn't work, doing a reinstall is a completely acceptable solution.

My suggestion on the TreeView is to use a bare bones sitemap and comment out as much of code as possible until you've isolated the exact line that is causing it. That's about the most common, most generic error around, so it's just a matter of trial and error to find out what is throwing it.
# February 27, 2006 12:20 PM

OldManChild said:

Thanks once again Scott,If you say so then I am at peace with myself.Will take your suggestion and see how it goes.
# February 28, 2006 7:03 AM

Phreshview said:

Any news on how to configure ASP.NET 2.0 to NOT restart the AppDomain if files change? I would prefer to not have to relocate my logs, files, temp, and xml directories outside of my web application. Anyone heard of the web.config setting "numRecompilesBeforeAppRestart"? What I would like is the setting "numFileChangesBeforeAppRestart" which I would set to infinity, because I change files all the time.
# March 10, 2006 8:44 AM

Scott Forsyth said:

Hi Phreshview. I don't believe there is a solution for it at this time. That appears to be the offical word from MS. A numFileChangesBeforeAppReset property would be a great idea. The other thing that would be great is to be able to specify which folders will cause the app reset and have all others be ignored.
# March 10, 2006 3:28 PM

Martin Bischoff said:

Hi, I also had the problem that my web app was restarted whenever a folder was deleted (deleting files and folders is an important part of my application's functionality).

After searching for a solution and contacting some people at microsoft I learned that there is a workaround: put your files/folders outside of the web app and create a file system link/junction from inside the web app to the external directory. FileChangeNotifications do not follow links/junctions.

You can read more about that workaround in Todd Carter's blog: http://blogs.msdn.com/toddca/archive/2005/12/01/499144.aspx
# March 13, 2006 8:24 AM

Max Mulawa said:

Very good blog entry.

I was developing my first app in ASP.NET 2.0, and I was wondering why I'm constanly loosing session. Now I know.
# March 15, 2006 6:10 PM

Hugo Augusto said:

I have an application with a filemanager and now I get the session timeout behavior whenever a directory gets deleted/renamed! What im curious is how DotnetNuke solved this problem since it uses also a filemanager. Anyone knows what is DNN using to get over this "bug"?

# September 23, 2006 3:35 PM

Tomas Petricek said:

Hi, thanks for your post :-) It helped me a lot when I was trying to find out what's wrong.

I submited this as a bug to MS Connect (https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=240686) to be sure that this will be improved in the next version of ASP.NET.

# November 24, 2006 6:59 PM

alam said:

if you change a file or folder application is recompiled and stays in the same appdomain. in one appdomain 15 recompiled are allowed if there is 16th changes the appdomain recycles and the counter resets to 0.

http://blogs.msdn.com/tess/archive/2006/08/02/686373.aspx

# November 29, 2006 5:18 PM

Sam said:

This is extremely annoying; I actually went back to 1.1 because of this. A reasonable solution would be for Microsoft to allow for a config file that specifies which dirs/files cause a restart rather than just restarting at any change. It's actually unbelievable that they would even consider something so stupid.

If a config solution ever becomes available will you post it here? Thanks for the info btw – although I knew about it, it’s good to see that others are pushing for a solution as well.

# December 25, 2006 4:17 PM

Curt said:

Scott,

Sorry for letting off steam on your blog.

I still need to get my app to work.  I tried changing files in the App_Data folder and it still kills the app.

Am I correct to assume that the only solution right now is what Martin Bischoff posted re http://blogs.msdn.com/toddca/archive/2005/12/01/499144.aspx (which I think is a non-solution for all of those small ASP.NET 2.0 apps that are hosted)?

# January 6, 2007 12:47 PM

Roy said:

I am having an issue where my session state is lost on my shared hosting environment.  Locally, I log in and my session remains active.  However, on the my shared hosting site, not long after I log in and store a session variable, I seem to lose the session variable fairly quickly.

My application does allow the uploading of files to the web server, however, the site is not launced yet and no one is using it.  I don't upload any files.  I only log in and go to a page over and over again that checks for my session state.  Usually in under 5 clicks, it seems to lose my session info and redirects me to the login page.

Any ideas?

On an unrelated question, it seems that when I deploy my ASP.Net 2.0 application, I have to copy my .CS (code behind) files to the web hosting server in order to get it to run correctly.  This was not necessary in ASP.Net 1.1.  Only the compiled DLL files were needed.  Is this something new in ASP.Net 2.0 or is it something wrong with my configuration?

Thanks for your help.

Roy

# February 6, 2007 9:45 AM

Leena said:

Compiled Solutions / workarounds so for

Sol 1: Use out of process session state.

Sol 2: Use Directory Junction between seperate Web folder and content folders

Sol 3: Disable FCNotifications in ASP.NET2.0 by adding DWORD FCNMode =1 under

HKLM\Software\Microsoft\ASP.NET key

Sol 4: Use .NET framework 1.1.4322 in side by side

Most of the time solution 4 seems to be easy way out till MS releases fix for this problem.

# June 1, 2007 3:23 AM

Kritter said:

I've found another solution and posted it to Microsoft's workarounds list: connect.microsoft.com/.../Workaround.aspx

I believe it is the best solution for the moment

# July 23, 2007 9:56 AM

Jerbear's Lair said:

In a previous post , I talked about Alternate Data Streams and how some virus-scanning engines use them

# September 28, 2007 6:39 PM

Techy News Blog » Virus Scanners and ASP.NET apps don’t always mix said:

Pingback from  Techy News Blog » Virus Scanners and ASP.NET apps don’t always mix

# September 28, 2007 6:55 PM

MSDN Blog Postings » Virus Scanners and ASP.NET apps don't always mix said:

Pingback from  MSDN Blog Postings  » Virus Scanners and ASP.NET apps don't always mix

# September 28, 2007 9:08 PM

» Virus Scanners and ASP.NET apps don’t always mix MSDN Blog Feed said:

Pingback from  » Virus Scanners and ASP.NET apps don’t always mix MSDN Blog Feed

# December 30, 2007 3:30 AM

» Virus Scanners and ASP.NET apps don’t always mix MSDN Blog Feed said:

Pingback from  » Virus Scanners and ASP.NET apps don’t always mix MSDN Blog Feed

# January 12, 2008 12:01 AM

Airline Travel » Virus Scanners and ASP.NET apps don’t always mix said:

Pingback from  Airline Travel » Virus Scanners and ASP.NET apps don’t always mix

# February 1, 2008 8:07 AM