Some techniques for better managing files in VS 2005 Web Projects

I’ve seen several questions come up in the last few days regarding techniques for structuring project layouts with web projects in VS 2005.  In particular, I’ve received several good questions asking for ways to manage large image directories as well as temporary storage directories underneath web project roots.  One goal with both questions has been to find ways to avoid these files from showing up in the VS web project solution explorer and cluttering up the web project view, as well as to avoid having these files/directories be prompted to be added to source control during check-in (note: you can just unclick having these files added to source control – but having to-do so can be a pain). 

 

This blog post covers some techniques for handling scenarios like these.  Prior to reading further, I’d strongly recommend you read my previous post on managing IIS web projects with VS 2005: http://weblogs.asp.net/scottgu/archive/2005/08/23/423409.aspx.  The section on virtual directory mappings and how they are treated will be particularly pertinent to some of the techniques below. 

 

Note that the final release of VS 2005 will have “Exclude File from Build” support to allow you to exclude/include code from being part of a build or publishing operation.  The below techniques are about going one step deeper and having files disappear entirely from the solution explorer, as well as to discuss techniques for sharing common directories across multiple projects.

 

Scenario #1: How to manage a 5,000 item /images directory in a web project

 

A really good scenario that someone sent my way today was one where they currently have an ecommerce solution that has a 5,000 item product catalog.  As part of their solution they have an “images” subdirectory underneath their web root that stores an image for each of the 5,000 products on disk. 

 

When they open up their web project using VS 2005 they are worried about having two problems:

 

1) They will have a 5,000 picture sub-directory in their heavily developed web project which will add clutter and slow down open/copy operations.

 

2) Managing this directory with source control will be a real pain – when you check-in or check-out of Visual Source Safe it will verify that these files haven’t changed or have been added to source control which can really take awhile.  Ideally there would be someway to manage these image files elsewhere.

 

A technique to make this experience much better:

 

For scenarios like these involving directory resources, one technique that you can easily use to better manage these items in web projects is to mark these directories as “virtual directories” in IIS.  This does not require any code-changes – it is purely a configuration setting you can manage via IIS.

 

Making these directories “virtual directories” provides four big benefits:

 

Benefit #1: Sub-directories marked as “virtual directories” in IIS will automatically be excluded from showing up in a VS 2005 web project.  A marker node will still remain in the solution tree to indicate that a virtual sub-directory is present, but no files or sub-directories will be added – and the virtual directory and all of its contents will be excluded from all source control and publish deployment operations.

 

For example, the below images show the difference when the “images” subdirectory is marked as a normal directory:

 

 

And here is the same web project again when the “images” directory is marked as a “virtual directory” in IIS.  Note that there is still an “images” node in the hierarchy to denote the presence of a virtual sub-directory – but no content is included underneath it:

 

 

Note that the content of the virtual-directory will be automatically excluded from all deployment operations, and will not be included in the source control for the web project (even in cases where the "images" virtual directory is physically stored underneath the web project on disk).

 

Benefit #2: You can now store these directories anywhere on disk.  If you want, you can continue to store them physically underneath the same web app or site directory.  You can also then optionally store them elsewhere on disk.  This can make some deployment scenarios much easier – since you can now manage these directory trees separately and update and deploy their roots independently.  Note that virtual directory views within the solution explorer will be the same regardless of whether the physical images subdirectory is under the project directory or stored elsewhere – it is completely transparent to the active project.

 

Benefit #3: You can have multiple web projects share the same virtual directories.  This can be a very useful and powerful technique when you have shared images, CSS or script libraries that you want to re-use across multiple web sites or web applications (for example: www.site1.com, www.site2.com, www.site3.com, etc).  One of the cool new features in the VS 2005 WYSIWYG designer is that it will follow virtual directory paths and load images, CSS files and scripts correctly even when they are stored in virtual directories.

 

For example: consider a scenario where you are building 3 different sites or applications where you want to share a common CSS design, and common corporate logo images.  You could lay out your application structure like this if you wanted to:

 

C:\CommonVDir\Images

C:\CommonVDir\CSS

C:\CommonVDir\Javascript

 

C:\Sites\www.site1.com

C:\Sites\www.site2.com

C:\Sites\www.site3.com

 

You could then map a virtual directory underneath each site to point to the “CommonVDir” shared library directory.  Not only will this work at runtime, but when you create a new page underneath the www.site1.com web project and reference the “commonvdir/css/style.css” stylesheet – the VS 2005 web designer will automatically pick up the styles and show things correctly in design-view, even though the actual stylesheet is never checked in or managed in the www.site1.com web project.

 

When you update a single CSS stylesheet under the c:\CommonVDir\CSS directory, it will immediately show updated in all 3 of the sites -- both at runtime and design-time.

 

Benefit #4: You can optionally create additional web projects to manage your various virtual directories.  This allows you to manage these separate virtual folders using their own development/design team, web project and source control settings.  For example, the below screenshot shows one solution with both the web application open, as well as a second and third web project open to manage the “images” and “styles” virtual directories as separate isolated projects (which could be checked-in separately under source control):

 

 

Scenario #2: How to dynamically generate and handle temp files in a web project

 

Another scenario that a few people have asked about is one where they are dynamically generating XML files underneath their web project at runtime, and they want to avoid having these files appear in their solution explorer and source control, and avoid having them be deployed by Visual Studio’s “Publish Web” feature. 

 

A technique to make this experience much better:

 

For temporary file storage that is dynamically generated underneath a web-project or web-root, you might want to consider using hidden folders. 

 

There are two benefits of this approach:

 

Benefit #1: Hidden files and folders are automatically excluded from the web project view under solution explorer, and will be excluded from all source control operations. 

 

For example, here is an example web project with a “tempstorage” directory containing XML files:

 

 

Here is the same web project except that the “tempstorage” directory has now been marked with a “hidden” bit:

 

 

Benefit #2: IIS will by default block requests for content stored in hidden folders.  This can be a useful security check to help prevent temporary files from accidentally getting downloaded by unauthorized users. 

 

The below two lines of sample code demonstrate how to dynamically create a new temporary directory, and then mark it as hidden using the System.IO namespace:

 

        Directory.CreateDirectory("tempstorage");

        File.SetAttributes("tempstorage", FileAttributes.Hidden);

 

You can also obviously just create the folder on the file-system yourself using Windows explorer, pull up its properties window, and click the "hidden" attribute too.

 

Note that whether a folder is hidden or not will not change any code or logic when accessing/writing files underneath it. 

 

Important Note: For maximum security, I would always recommend that you not grant any write access to the directory underneath a web project or application root.  If you need to store temporary files somewhere, it is far more secure to store them in a totally non-accessible location.  I’ve only included the above section on temporary storage because several people have asked about it.

 

Scenario #3: How to store non-deployed files in web projects

 

A final scenario a few people have asked me about is how they might be able to store files within web projects and under source control that can be used during development time (for example: design-docs written in word or visio, or adobe photoshop .psd files for layered images), but will be automatically excluded from any deployment or publishing steps. 

 

A technique to make this experience much better:

 

ASP.NET 2.0 supports a concept of “build providers”.  These are classes that implement the System.Web.Compilation.BuildProvider base class contract, and which can participate in build-operations both in VS 2005 at development-time and ASP.NET at runtime.  Developers are free to build and implement their own BuildProviders to add their own custom semantics to processing files within ASP.NET.  For example: you could create an .ORM file extension for files that contained XML to declaratively represent an OR mapping database relationship.  Your provider could then dynamically generate strongly-typed classes that were included in the web anytime one of these files was added to the project – and you would get both intellisense within VS 2005 at design-time as well as full runtime support.

 

One of the built-in build providers in ASP.NET 2.0 is a provider called the “IgnoreFileBuildProvider”.  Its semantics are that it ignores whatever file extension is mapped to it, and that it will also automatically prevent that file from being deployed during a publish-web or compilation operation.

 

Developers can then use this feature to add extensions to the web.config file in their local web projects and effectively block any file-type they want from being deployed by VS.  For example, if I wanted to prevent all .doc (word), .psd (photoshop) and .vsd (visio) files from ever being deployed, I would add the following section to my web.config file:

 

<configuration>

  <system.web>

     <compilation>

      <buildProviders>
         <add extension=".doc" type="System.Web.Compilation.IgnoreFileBuildProvider" />
         <add extension=".psd" type="System.Web.Compilation.IgnoreFileBuildProvider" />

         <add extension=".vsd" type="System.Web.Compilation.IgnoreFileBuildProvider" />
      </buildProviders>

     </compilation>

   </system.web>

</configuration>

 

I am then free to add and check-in these file types anywhere in my project if I so wanted to – and they can be used/modified/accessed throughout the development lifecycle – but will be excluded anytime a build is produced.

 

Hopefully this helps show some useful techniques you can leverage.

 

- Scott

Published Friday, August 26, 2005 1:10 AM by ScottGu

Comments

# re: Some techniques for better managing files in VS 2005 Web Projects

Friday, August 26, 2005 11:07 AM by Bob Archer
Scott,

Thanks for the last three blog articles. They have really cleared up alot of the questions and concerns I have had about web projects, er, web site projects, er... you know what I mean.

One question on the above. When you say "virtual directory" I assume you mean one with an "application" tied to it? Or can it be a virtual without an application.

One common theme I seem to see is that to get the most flexiblity out of web projects you have to use IIS based projects.

The more I see on the new web project system to more it seems that the file system type which uses the cassini server isn't really practical for anything more than a smaller or hobby type site.

Is this the case? If not, what am I missing? Will I be able to use IIS based sites and run least priveledge user. What type of permissions do I need to use IIS based web projects?

Thanks,
BOb

# re: Some techniques for better managing files in VS 2005 Web Projects

Friday, August 26, 2005 11:08 AM by Julich
All these techniques seem to be very effective and simple to implement. Thanks a lot Scott!

# re: Some techniques for better managing files in VS 2005 Web Projects

Friday, August 26, 2005 1:13 PM by Bob Archer
Stuart,

Doesn't it store the web path in the .webinfo file?

# re: Some techniques for better managing files in VS 2005 Web Projects

Friday, August 26, 2005 1:33 PM by Stuart Ballard
Bob, you're a genius! I'll have to do some testing but that looks very promising :)

It's in the .suo file too, but since we don't create an suo file at all, it'll take it from the webinfo if I create one. I'll let you know :)

# re: Some techniques for better managing files in VS 2005 Web Projects

Friday, August 26, 2005 1:38 PM by scottgu
Hi Stuart,

I'm glad it is starting to win you over. ;-)

Unfortunately at this point we aren't going to be able to support app_code underneath non-root directories. One thing you could do, though, would be to split out your your project organization something like this:

c:\mysite\common <= class library of common biz/dal logic
c:\mysite\wwwroot <= root site application
c:\mysite\wwwroot\store <= store application under site
c:\mysite\wwwroot\app2 <= app sub-application

This would allow you to have separate app_code directories underneath the store and app2 applications -- with all three web projects linking to the common class library project for things shared across the solution.

BTW -- I've got mail off to a project system expert on the VS 2003 question and will let you know what he gets back to me with regarding the SUO file.

Hope this helps,

Scott

# re: Some techniques for better managing files in VS 2005 Web Projects

Friday, August 26, 2005 1:44 PM by Mayur
We usually "Exclude" such folders from the project which IIS still can access.

This works as long as you have static content (images, .js etc.) in such folder.

# re: Some techniques for better managing files in VS 2005 Web Projects

Friday, August 26, 2005 5:17 PM by Alexey Kouzmitch
Hi Scott,

Great suggestions. Would the hidden files be visible when "Show All Files" option is selected? That would be dy-no-mite.

Thanks.

Alexey.

# re: Some techniques for better managing files in VS 2005 Web Projects

Saturday, August 27, 2005 1:02 PM by Stuart Ballard
I realized I'm still missing something very important about what the new system actually is. Perhaps I've missed it in one of your posts or haven't read the right documentation, or maybe another post is needed to clarify. Or maybe it's simple enough to just explain in a comment :)

Basically, I realized I don't actually understand what App_Code *is*. My understanding up till now has been "it's where your C# code files go, except for codebehinds". But with the discussions of build providers and the like, I'm not clear on exactly what this means.

Specifically, what exactly happens if you simply put a .cs file outside of App_Code?

With all the discussion of build providers, it seems natural that there would simply be a "CSharpCodeBuildProvider" applied to ".cs" files. But we've established that build providers can't be specified on a per-folder basis, so what makes App_Code special? And since build providers seem to just be classes, would it be possible to *write* a build provider which would let C# files outside App_Code be treated as if they were inside it?

I realized that without understanding this I really can't form an educated opinion on how necessary it is to mess with my file structure...

Another thing I wonder is whether you could get around the "no folder-specific build providers" rule by writing a custom build provider: Write a build provider which reads the web.config for the subfolder, looks for a <buildProviders> directive (or perhaps a <myBuildProviders> tag if <buildProviders> is illegal outside the top level), instantiates the provider mentioned there, and delegates all other methods to it...

Would IgnoreFileBuildProvider still work if its methods were being delegated to by another provider, or is its name hardcoded somewhere?

In reference to your proposal, unfortunately using separate projects to hold the different folders wouldn't work unless it's possible to instantiate user controls from one project inside another. In 1.1 this was forbidden, and I can't see how that would have changed, as the code lives in different appdomains...

(Thanks for looking into the .suo file thing. So far, the webinfo file trick seems to be working, although issues in other parts of the script have thus far prevented me from testing it properly)

# re: Some techniques for better managing files in VS 2005 Web Projects

Saturday, August 27, 2005 5:23 PM by Stuart Ballard
Bob, Scott,

Unfortunately the .webinfo solution does *not* work. It is possible to use "devenv foo.sln /rebuild debug" without having to answer that question, but it's then impossible to open the IDE to that project: you'll be prompted for the web location, and then forbidden to actually pick the location you want because it "already contains a project" (of course it does, it's the one you're trying to open!). It appears that even though this file contains exactly the information VS.NET is prompting for, it won't read the file to see if it can get it without a prompt.

It must be coming from the .suo file after all...

# Not only managing interest me after this post...

Sunday, August 28, 2005 1:01 PM by LukCAD
Hello Scott,
Nice thing to configure "build providers" in VS project, i din't know about it. Now it will save me a lot of time.(I usually never do it early and sometime i losted my all disigned .psd). It is great feature foe VS2005.
But I still has old problem that i had early when i am downloading new files to web server but their name was been old. Explorer can't feel the new files in this case, and i use not good way: i make new name of images (to provide instant changing and avoid their caching on client side) by expanding name of image the chare "?" and then value from System.DateTime.Now or System.DateTime.Ticks. But may be is more civilization method to say to explorer that in the same named file is more new information and IIS or IE must skeep cache and get new pics (with old name) from server but not from client.
I feel that i don't know a lot, but still not found the answer for that may be simple question.
Sincerely, LukCAD.

# re: Some techniques for better managing files in VS 2005 Web Projects

Tuesday, August 30, 2005 7:55 AM by Dirk Rombauts
Hello Scott,

Your posts over the last few days have been very informative - keep them coming!

Those paragraphs about BuildProviders really sparked my interest. Is it possible to use this technique in winforms applications too, or is it reserved for web apps exclusively?

# re: Some techniques for better managing files in VS 2005 Web Projects

Tuesday, August 30, 2005 10:40 AM by scottgu@microsoft.com
Hi Dirk,

BuildProviders are indeed pretty interesting. One of the nice things you can do with them (besides excluding files from deployment) is perform conversions of files at build-time and compile them as part of the assembly. This allows you to-do really interesting things with XML and other file-formats that you want to use to declaratively specify things.

Unfortunately WinForms projects don't support these just yet. You could write a MSBuild action to perform a pre-conversion to source to accomplish something similar though.

Hope this helps,

Scott

# re: Some techniques for better managing files in VS 2005 Web Projects

Tuesday, August 30, 2005 4:41 PM by Steve
We create applications under one root application by creating a new web solution.

ie.
Main web app
c:\inetpub\wwwroot\MainApp

Then, let's say I want to make a webstore.

c:\inetpub\wwwroot\MainApp\WebStore

Then we click 'remove' application in IIS - it appears as a folder instead of an application - using the webconfig, etc.. from the root.

This makes fixes to WebStore easier because the code is in WebStore.dll not MainApp.dll.



This is 1.1. Will this senario work ok with asp 2.0 ?

# re: Some techniques for better managing files in VS 2005 Web Projects

Tuesday, August 30, 2005 9:16 PM by scottgu@microsoft.com
Hi Steve,

I think you can continue doing this with V2.0 -- the one question I have is what tool you are using to edit your application and what settings you are using?

If you have VS 2003 I think it will always generate your assembly in the \bin directory of the application root you opened it under -- so in your case above that would be c:\inetpub\wwwroot\MainAp\Webstore\bin. If the WebStore directory isn't marked as an application - then your assembly loads would fail with ASP.NET V1.1. Or do you copy them elsewhere?

Thanks,

Scott

# re: Some techniques for better managing files in VS 2005 Web Projects

Thursday, September 1, 2005 12:27 AM by Steve
Thank you .

We use VS2003

We create a reference to the project to WebStore in MainApp.

When we build MainApp, it puts the WebStore.dll in MainApp.

(Yes , webstore also has a bin, but that is not needed when moving the code off the development machine)

In VS2005, I hope to get away from needing to do this.

# re: Some techniques for better managing files in VS 2005 Web Projects

Wednesday, September 7, 2005 3:15 PM by do we have to to make that virual directory thing
for example if i have a project say

c:\inetpub\wwwroot\one

then i have a sub folder inside one

c:\inetpub\wwwroot\one\two

but images folder which are also used on
the pages inside folder two is inside main
folder one. do i have to make that virual
folder thing inside folder two as well ?

# re: Some techniques for better managing files in VS 2005 Web Projects

Monday, September 12, 2005 7:20 AM by Rocky Moore
I for one will greatly miss the "Show All Files" option as I used this heavily. It was very simple and with a single click of a button would make my files that were considered hidden to the project to either appear or disappear.

Would be great if that was bought back to "show/hide" hidden files inside the solution explorer!

One point I am curious on though, with this new method, how do you maintain a local and server copy of the web.config file (and possibly other files) where values may be different depending on if it is local or on the production server?



# re: Some techniques for better managing files in VS 2005 Web Projects

Wednesday, September 27, 2006 6:39 PM by xyz
I have a problem with images.My images are in a folder under my project For example:c:/my project/images Inside my project i have several folders and webforms which use images these images dont work with the file within the folders. What is the best way to fix these.Is this technique of virtual directory a feature in vs2005 or was it there earlier in vs 2003.please let me know

# re: Some techniques for better managing files in VS 2005 Web Projects

Friday, February 2, 2007 6:54 PM by iso

i wanna webconfig file not to be deployed based on this   this should take of this but I don't  see it workin g???

 <compilation  defaultLanguage="c#" debug="false">

     <buildProviders>

       <add extension=".config"

        type="System.Web.Compilation.IgnoreFileBuildProvider" />

     </buildProviders>

   </compilation>

# re: Some techniques for better managing files in VS 2005 Web Projects

Wednesday, March 7, 2007 12:54 PM by Diego

Hi Steve -

I work in an organization where getting things like Creating an application of a directory in IIS is very hard to have done. What we do have is the root application and i am working on convicing my security office to allow the .net machine account on the web server read/write access to the root app_code folder....

If all that is available is the root app_code folder for shared code, is it prudent to create directory structures within this folder to represent classes and objects needed for any "sub-applications" ?

For example, i may create a web form that needs to use a dataset object at root$\dir1\dir2\myapp. Instead of making "myapp" an application in IIS and then using VS to develop this app with it's own app_code and web.config, I would create the web form on the root web application and use the root app_code as follows: root$\app_code\dir1\dir2\myapp\mydataset.xsd...

My question is, would this ultimately cause major performance issues on the web server? If i add this dataset to the root app_code folder, will the web server then recompile every class and object in the root app_code folder?