Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

September Update: For more details on how to optimize web project build performance in VS 2005, please review this blog post I did here. 

Symptom:

When building a VS 2005 Web Site Project you experience very slow build times – where compilation seems to mysteriously pause for 30+ seconds (or even minutes), before continuing each time you do a build or F5 of the project.

This compilation performance slow-down might be the result of a problem I call “dueling assembly references”.  This can occur when you setup multiple “automatic refresh enabled” file-system assembly references within your web site project to multiple assemblies that in turn share common assembly dependencies whose version numbers are different.  The post below describes this in more detail and how to fix it.

Some background on how file-assembly references work:

Developers using VS 2005 Web Site Projects can add references to assemblies in multiple ways.  When you right-click on a project and choose the “Add Reference” menu option, it provides a dialog like the one below that allows you to reference .NET assemblies registered in the GAC, class libraries built as projects in the same VS solution, COM components, and file-path based assemblies that you pick using the “Browse” tab of the “Add References” dialog:

When you add a file-system based assembly reference using the “Browse” dialog above, VS 2005 Web Sites will copy the assembly picked into the \bin directory of the project as well as (by default) add a .refresh file into the \bin directory that contains a relative path string that points back to the original assembly path location:

When you build the solution, VS will automatically check those assemblies with .refresh files to see if a newer version of the assembly is available, and if so automatically re-copy the assembly and re-compile the solution using it.  This avoids you having to manually update the assembly yourself every time it changes (it also avoids you having to check-in the assembly under source control).  If you don’t want this “automatic refresh” capability, you can delete the .refresh file – in which case VS will not check the timestamp nor update the assembly for you.

Because assemblies often have dependent assembly references, Visual Studio will also automatically copy dependent assemblies that a reference assembly requires into the \bin directory of the target web-site as well.  For example: if you setup a file-based reference to ClassLibrary1.dll, which in turn uses an assembly called SharedLibrary.dll that resides in the same directory, Visual Studio will make sure both assemblies are copied into the Web Site’s \bin directory (note: if it didn’t do this then the app would fail to run).

Dueling Assembly Reference Problem

The problem I refer to as a “dueling reference” occurs when you setup multiple file-based references to assemblies from a VS 2005 Web Site Project that are each updated dynamically (using .refresh files), and which in turn have dependencies on different versions of a common shared assembly library.  

For example, assume your web-site has a reference to AssemblyA.dll and AssemblyB.dll – which in turn each reference AssemblyC.dll.  This will work without problems if the referenced AssemblyC.dll is the same version for both AssemblyA.dll and AssemblyB.dll.  But it will cause problems if the AssemblyC.dll being used is different between the two:

In cases where AssemblyC.dll is different, VS ends up copying the AssemblyC.dll file twice for each build – since it continually thinks that the assembly has been updated (once for each reference).  This ends up requiring all references to be recalculated by the compiler, and a full re-build to occur within the IDE.  This will cause build performance to slow down dramatically, and will cause builds to appear to pause as VS does this reference recalculation and fix-up on every single build.

How to Fix This Problem

There are a couple of ways to fix this problem:

1) The “most correct” way to fix this issue is to make sure your class library references are built against the same version of any dependent assemblies.   This is good to-do not just to fix the above build performance problem, but also because it decreases the likelihood of introducing hard to figure out bugs in your application (at runtime only one version of the shared assembly is going to be used –so if you don’t fix this at least one of your dependent assemblies will end up running against an assembly it wasn’t built/unit-tested with).

2) The “easiest quick fix” way to resolve this issue is to modify one or more of your assembly file-based references to not be “automatic refresh enabled”.  You can do this by deleting the .refresh files within your \bin directory that produce that shared conflict.  This will avoid having VS auto-update the assemblies, and so will prevent the dueling update conflict altogether.   You might find it useful to quickly disable this behavior by deleting the .refresh files as a stop-gap, and then re-enable the auto-refresh behavior once you fix the shared assembly conflicts.

Note that VS 2005 Web Application Projects don’t have this dueling reference issue (since they don’t use .refresh files for file-assembly references).  So you won’t have this build-performance problem with it (instead it will just pick one version of the shared assembly to use).   However, you still want to be careful about cases where you have two components built against two separate versions of a shared assembly – since this can still cause hard to understand behavioral bugs at runtime that end up bypassing your carefully written unit-tests (which were run against a different version of the shared assembly).

Hope this helps,

Scott

Published Sunday, July 30, 2006 1:35 PM by ScottGu

Comments

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Sunday, July 30, 2006 5:58 PM by Eric
Good to know... Is there going to be anything in SP1 that will address this from the VS side? More importantly, when are we going to see SP1?

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, July 31, 2006 1:56 AM by Matt Tester
I had a similar problem with the Enterprise Library assemblies I had referenced. Removing the '.refresh' files worked great ... but it just doesn't seem right to have to do that :-) Yeah, I would also love to know when SP1 will be released ... will it include the "VS 2005 Web Application Projects"?

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, July 31, 2006 4:45 AM by Pawel Pabich
Unfortunately that’s not my case. The compilation time is still about 2 minutes. It takes 2 minutes only when I change one of the library projects(there are 7 of them), even if add only an empty line. The most interesting things is that all library projects compile very quickly only the web app compiles ages. Do you know why?

# How do I find out which versions are being used at run time.

Monday, July 31, 2006 1:51 PM by Gurpiar
Scott, I remember I had VS having this problem, it was hanging for minutes, it was refreshing files from a different machine on the network. I remember I tried to delete one of the .refresh file, but it was automatically getting regenerated on recompile, I don't know why was that happening. Anyway, My real question is, how can I find out which version of files are being used at run time?

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, July 31, 2006 1:54 PM by Danny
Thanks Scott. I was actually browsing your site while waiting for the 30+ seconds for my web site to build!

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, July 31, 2006 1:54 PM by Gregor Suttie
Thank you thank you thankyou - I have been tearing my hair out with a project which takes a ridiculous length of time. Once again thanks. Cheers Gregor

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, July 31, 2006 9:52 PM by ScottGu

Hi Eric,

I believe the plan is to add output text to the output window on builds that calls out when an assembly is being refreshed -- so that it is obvious what is going on when this situation happens (since right now the pause is unfortunately a black box).

In the situation when this happens VS can't really know the "right thing" to-do in terms of guessing as to which assembly to use, so it isn't something that can be totally fixed on the VS side.  What we are trying to-do with the message is make it clear what is happening, and help provide developers with a way to understand the situation (them linking to two libraries with mis-matched shared assemblies) and how to best fix it within their code-base.

In terms of the overall final date for SP1, it is something that I know the team is working on very hard right now.

Hope this helps,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, July 31, 2006 9:55 PM by ScottGu
Hi Matt, Yep -- VS 2005 SP1 will have VS 2005 Web Application Project support built-in. Note that you can download it today now, though, and not wait for SP1. PSS is also "fully supporting" the download -- meaning you can call up for product support and receive QFEs if necessary for it. Hope this helps, Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, July 31, 2006 9:56 PM by ScottGu

Hi Pawel/Ryan,

If you want to send me email (scottgu@microsoft.com) about your project with the 2 minute delay, I can help figure out the delays you are seeing.

I am also planning on writing a new post later this week on "Tips/Tricks to Optimize Build Time" that I will post here with a list of suggestions.

Thanks,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, July 31, 2006 9:57 PM by ScottGu

Hi Gurpiar,

At runtime ASP.NET will load the assembly in the \bin directory of the application.  So ultimately that is the version that ends up being used at runtime.

Hope this helps,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, July 31, 2006 9:59 PM by ScottGu

Hi Lakshmi,

As you add more assemblies into the \bin directory, ASP.NET will load a little slower on first run (which is what you are seeing when you first debug).  This is because ASP.NET is first shadow-copying the assemblies to a different directory and then loading them from there (so that they aren't locked on disk).

One thing you should look at is whether you need all 52+ assemblies, and/or if there are some shared assemblies that are best referenced from the GAC.

Note that at runtime you won't have perf issues because of too many assemblies in \bin -- but it will impact the perf the very first time you load the app.

Hope this helps,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Tuesday, August 1, 2006 12:58 AM by Gary
Thanks for this article Scott. My build times are also very slow I have 22 referenced DLL files and only one of those has a 'refresh' file, I deleted it and it is still just as slow. The current solution is to use the excellent web application extension however the 'web site' option has features which we must use for release deployment... if only it was as fast as 'web application' it would make debugging a dream. Thanks, Gary

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Tuesday, August 1, 2006 5:19 AM by Poppyto
Hi Scott, I removed a .refresh in my bin folder but it does not speed up the compilation time. I have 100 files (aspx & aspx.cs) and compilation takes 2 minutes. Maybe it's a good time ? But it's very long for my brain :) ! Anyway, I hope there will be great improves on compilation time with a future SP... As many developers, I remember that compilation time in vs2003 was better ;)

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Tuesday, August 1, 2006 11:44 AM by Ran Davidovitz
Just for the record our solution takes 85 seconds from pressing the rebuild (only for the web site). * we have 3 developer that are doing rebuild lots of times during the day (think of the waste)

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Wednesday, August 2, 2006 2:11 AM by ScottGu

Ran/Popyto,

One other thing I'd recommend checking is whether your web.config file has disabled batch compilation within ASP.NET (this is done via a batch attribute on the <compilation> section).  When this is set to false, it will signficiantly impact compile-time performance and is something else to check.

I'll be doing a blog post later this week (or weekend) that provides other suggestions on how to optimize build performance and should help you further.

Thanks,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Wednesday, August 2, 2006 2:12 AM by ScottGu

Hi Gary,

You mentioned above that you need to use the web-site project option for features that are needed for release deployment. What specific features do you need that isn't provided by the VS 2005 Web Application Project option?

Thanks,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Thursday, August 3, 2006 4:24 AM by Martin
Thanks for the informative post. Are there similar problems with solutions that have a lot of project-to-project references (e.g. website1 references libraries A and B which both reference libaray C and so on)? We have a solution containing 4 web site projects and about 20 class library projects. There are a lot of references between these projects and build times could be better. Martin

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Thursday, August 3, 2006 9:31 AM by Bruce
You mention that compilation seems to pause for 30 seconds or so. I am not sure if this is the same symptom, but I often get the rebuild successful notice in the statusbar, but shortly after that my screens, mouse, etc all seem to freeze or operate very sluggishly. The solution tree seems to blink in and out. This continues for 15-30 seconds. Looking at the task manager, I can see that DevEnv is doing something on occasion, but it certainly is not any more than 40% CPU. Is this also dueling assemblies? This is not a web project, and I don't have any .refresh files, although I have multiple libraries that are referenced by several applications in the solution.

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Sunday, August 6, 2006 1:10 AM by ScottGu

Hi Bruce,

Are you having this problem with VB projects?  There is a VB performance QFE that is now available that fixes a number of known performance issues.  This blog post describes the fixes and how to get them: http://blogs.msdn.com/webdevtools/archive/2006/07/24/677180.aspx

Hope this helps,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, August 7, 2006 1:15 AM by Bimmer
I have the same (I believe) issue as Bruse has. THe project is a C# WAP, with 5 dlls. No it is not that bad yet, ctrl-shift-f5, build complete, freeze for about 5 seconds. The weired thing is that it just started to happen one day, not something that progressivly getting worse. It also happend on a project that I worked with couple of month ago. The PC shouldn't be an issue, given P4 dual core with 2G ram. Thanks Scott for all the hard work!

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Tuesday, August 8, 2006 12:50 AM by ScottGu

Hi Bimmer,

Do you have any other utilities installed (for example: ReSharper or another 3rd party package)?  Some of the issues I've seen reported end up being the result of these utilities which might have issues.

Thanks,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Tuesday, August 22, 2006 2:49 PM by Curtis
We are also seeing a slight freeze/build hesitation when doing a rebuild all and the compilation hits the web project. All the other assemblies in the project are super fast. Tried deleting the refresh files and it seems to help a little, but not much. I don't think we have the dueling reference issue because most of our referenced assemblies are versioned and we only include projects in the solution for assemblies we are actually developing on. We probably have 15-20 external assemblies and 150 pages or so. We also tried installing a couple of the most recent hotfixes (one for .NET Framework, one for VS). This helped a bit with stability, but not really with compiling.

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Wednesday, August 23, 2006 1:31 PM by mitli
Thanks for the tip! cut down the compilation time down to 10 secs from a good 2 minutes. I know that basically means I have to sort out my assembly references - but atleast this bits sorted!

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Saturday, August 26, 2006 3:14 AM by ScottGu

Hi Curtis,

Can you send me email describing the situation more?  I can then help loop you in with folks on the team to investigate.

Thanks,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Saturday, August 26, 2006 10:34 AM by Denis

You saved a LOT of my time...

Thanks !

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, August 28, 2006 7:54 PM by Thomas S. Trias
What about "Projects" refernces? I notice that the Version ends up as "Auto Update", but there is no .refresh file and Update is not an option. I like to have web projects for both our development and QA sites in my solution, but I only want the QA site to auto update with the Release build of the assembly projects. Likewise, it would be nice if the development site only auto-refreshed the debug version. I guess I should use a file-system reference instead of a Project reference (although it would be nice to be able to tie the reference to a particular release via the config. manager). I also notice that if I delete the .refresh file, sometimes the assembly no longer shows up in the references (especially if it is referenced by anoth assembly). -Tom

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Tuesday, September 19, 2006 7:58 AM by RedHotSly
Hi, thanks for the tip. I after deleting the .refresh files, VS2005 thinks that the dll and pdb files should to be added to VSS. They all show up in the "pending check-in" window. I don't want theses files in VSS. Do you know how to fix this? Thanks -Sly

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Tuesday, September 19, 2006 2:13 PM by sgero
I removed \bin\*.refresh and F5 performance is back!! Thanks!

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Tuesday, September 19, 2006 11:59 PM by ScottGu

Hi Sly,

If you don't have a .refresh file associated with the assembly, then VS will assume you want to check-in the file into source control.  If you don't want this behavior, then you probably want to add back the .refresh files.  As long as you don't have a dueling assembly issue like above, there is no performance hit for having this.

Hope this helps,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Tuesday, September 26, 2006 11:30 AM by Ryan Shripat
Hi M.A.Meligy, I was having the exact same problem - when I renamed my atlas .refresh files, compilation time went down from 48 seconds to 25 seconds. Great tip, Scott, thanks.

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Monday, October 9, 2006 1:02 PM by wizardnet
Removing .refresh, works for me. Very early phase of my project just added 3 aspx pages and references to atlas control toolkit and Microsoft.Practices.EnterpriseLibrary, the build time it was up to 2 minutes, now is 10 sec. Thanks Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Wednesday, October 11, 2006 3:17 PM by Luke
Whew, thanks. This article fixed a major headache I was having.

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Thursday, October 19, 2006 4:17 AM by Linda
Hi, my project was working fine until yesterday suddenly start to build very slow.... I delete the .refresh and nothing happens... is still very very slow... does anyone has any ideas? thanks

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Thursday, October 19, 2006 11:55 PM by ScottGu

Hi Linda,

Did something change on your machine (did you install a new product, or anti-virus software, etc)?

I'd also recommend reviewing this post to learn how to speed up web site projects: http://weblogs.asp.net/scottgu/archive/2006/09/22/Tip_2F00_Trick_3A00_-Optimizing-ASP.NET-2.0-Web-Project-Build-Performance-with-VS-2005.aspx

Hope this helps,

Scott

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Thursday, October 26, 2006 6:21 PM by Jason
I think I became clinically depressed for about 1 week because it was taking me 2 minutes to build my web site each time… now it just takes 10 sec … You are a God Scott. Thanks

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Tuesday, October 31, 2006 9:55 AM by M6rk
Anyone know how to suppress the compiler warnings I now get now that i deleted the .refresh files? "Unable to get auto=refresh path for reference '...'"

# re: Common Gotcha: Slow VS 2005 Web Site Build Performance Because of “Dueling Assembly References”

Sunday, January 21, 2007 11:57 PM by It

Fantastic post. Removing the .refresh files reduced a 2 minute build to about 10 seconds.  However when i put them back the build was back to 2 minutes again even though all referenced libraries were of the same Assembly Version. I removed the two Web Sites from the Solution, compiled then re-added them and voila - it was down to 10 seconds again. I find "Web Sites" and their opaque reference structure incredibly annoying. I will move to Web Application Projects ASAP.