VS 2005 Web Project System: What is it and why did we do it?
There has been a lot of online discussion lately about the new web project system that is being introduced with VS 2005. My goal with this post is to explain a little more about what it is, and provide some background as to the features it provides and the motivation we had in introducing it.
What are Web Projects?
Web Projects are used when building ASP.NET applications inside Visual Studio 2005. They can be used either standalone where everything within an application lives inside a single project, or together with class library, data, web-service, unit-test and/or any other project supported by Visual Studio as part of a Solution. They fully support references and project build-ordering within a VS solution -- you set these up and manage these cross-project relationships using a .sln solution file just like you to-do today with VS 2003 (for example: you might have 3-4 class library projects that build class library assemblies that are then consumed and used by the web project).
When you are ready to “publish” a web-solution, you can use the new “Publish” option on the “Build” menu to compile and generate a deployable directory structure for a web-server. This is logically equivalent to the user workflow steps taken with a WinForms client application project in VS2005 (except its Publish option generates a client “Click-Once” package instead of web site). You can also automate web project build operations using the new MSBuild system (more on this later) for command-line and automated build server scenarios.
So What Are The New Features with VS 2005 Web Projects?
The new VS 2005 web project support provides a number of new feature enhancements over VS 2002 and VS 2003. A few of the big improvements that we think developers will like:
No More FrontPage Server Extension Requirement
Unlike previous releases of Visual Studio, VS 2005 no longer requires developers to use or have FPSE (FrontPage Server Extensions) installed in order to create or build web projects.
You can now browse, create, and open local IIS applications and vdirs without FPSE on the box (instead we now have support for VS 2005 to directly use the IIS Metabase configuration store to attach and identify sites/apps/vdirs and associated physical path bindings). You can also edit and open web projects directly off of the file-system, or access them remotely over FTP.
FPSE is still fully supported for those who want it. But if you are in the camp that doesn’t want it (which based on feedback is 95%+ of you), your _vti_ directories will be a thing of the past.
Built-in Development Web Server (No IIS Requirement & Debug as
In addition to providing full support for developing and testing web projects on IIS, VS 2005 also includes a built-in development web-server that ships in the box. This test web-server is ideal for doing development when IIS is either not installed, or your IT policy limits desktop deployment of it.
This web-server only handles local requests for security reasons (so people can’t access your application remotely while you are developing it), and it automatically shuts down when you close your web-projects. It also supports developers debugging applications running within it while running under a non-admin “normal user” account. Once you’ve built your application using it, you can then upgrade the web to either run under IIS locally or deploy the application to a remote server to test and verify. There is also now a new built-in “Copy Web” wizard that can help with copying your web to a remote server (it supports FTP and FPSE as copying protocols too).
Obviously we also fully support building web projects against IIS directly (note: this is required when you are building a solution with nested vdirs and sub-applications).
More Flexible File Management and Better Cross-Tool Collaboration Support
Web projects have a few unique characteristics that often differentiate them from other types of application projects:
1) Web projects are almost always built using a variety of very-different tools in parallel. Images are worked on using tools like Photoshop, Illustrator, and FireWorks. CSS and .ASPX/.HTM content layout is worked on using DreamWeaver or FrontPage. Code is worked on using VS. A tremendous amount of integration and collaboration goes on between these different types of resources. Sometimes the same person performs all of these tasks, typically for larger projects they are split up across multiple people working together.
One of the big complaints we hear with VS 2002/2003 is how hard it is to manage these cross-tool workflows – especially given that non-VS tools do not work with FPSE and do not integrate at all with VS project files. For example: a designer might update a CSS file in a web project to use a new background image, and add the new image to an images directory under the web project using their image editing tool. If the web developer using VS 2003 does not coordinate this change by updating his or her project file to remove the old image and add the new image to the project file manifest, then they will find that things work ok during development, but it will fail in production since the new image will not be copied/deployed when the web project is built to a new location.
2) Web projects are also unique in that the file layout of pages, images, CSS stylesheets, Flash SWF files and other items within a web application are directly surfaced to end-users accessing the running web app with a browser. File-layouts are almost always hierarchical, and a great deal of attention and work is done when coming up with this organization. The number of files and project items within a typical web site can easily climb into the hundreds or thousands of items. One of the complaints we hear with VS 2002/2003 is how hard it is to manage this within the IDE – especially when source control is used and it is necessary to check-out and then check-in a common project file every-time a file layout is changed or a file is added/renamed/deleted.
To better support both of these scenarios, VS 2005 web projects no longer store a list of all files in a central web project manifest file. Instead, the content and layout of the web project are directly inferred from the file-layout and file contents of the web-project folder. Additions/deletions/movements of files and folders within the web-project do not require updating one central project file that is a single point of contention. This means you can easily use any non-VS tool that directly works off the file-system with your web project, and that when running under source control you do not need to lock and update the project file anytime you move or make a file change.
The project layout under the Solution Explorer for web projects has also been changed to show a file-system view of the web project. This ensures that developers always see a 100% accurate view of what the exact file-layout and content of their web projects look like (something not true with VS 2003). Note: we no longer show the “references” tree underneath the root web project node because it is not represented in the file-system – instead you pull up the references list for the project from the project’s property-page (right click on the project in the solution explorer to quickly go there), and you can add a new reference to a project directly using the “Add Reference” menu or context menu item.
Web projects obviously fully support source control. If you move or rename a file in a project under source-control, VS 2005 will move and rename the item appropriately with the source control depot (preserving history) without requiring you to exclusively check-out/check-in a project manifest to update it as well.
Better support for building much larger web projects
We’ve heard complaints from developers who work on large web projects with VS 2003 that the IDE and workflow environment doesn’t scale well to handle lots and lots of files and content. Opening a large web-site can sometimes take minutes, and re-building and running it can often take a very long time when you make a change.
With VS 2005 we’ve tried to significantly improve this scalability to handle hundreds and thousands of items within a web-project. As an example, on my local desktop machine (a P4 3Ghz) I just tested opening up a local web project with VS 2005 that contains 9,000 files (5,000 .aspx pages and 4,000 images – distributed across 36 directories each containing 250 items). It opened with VS 2005 in under 3 seconds. I could then open up and immediately begin editing any of the .aspx pages in any directory without any pauses or delays. This is several orders of magnitude faster than any previous version of VS (where it would take minutes for a project of this size to even open).
Doing a first-time clean build of this 9,000 item web-site takes about 42 seconds on my machine (about the same time as VS 2003). What is cool, though, is that after building the web project once I can now open any of the 5,000 pages, make a change to the code in the page, re-build it, and immediately run and test it in less than 2 seconds from the moment I hit Ctrl-F5.
There were a couple of architectural changes that we made to facilitate this. One of the biggest changes was that we’ve moved away from compiling everything in a web project into a single DLL (which is the only mode that VS 2002 and VS 2003 support). Instead, we now partition the web project into multiple more granular assemblies.
All non-code-behind classes, datasets, web-services and other non-UI code that live in the web project are by default now compiled into one common assembly. Pages and User-Controls are then compiled into separate more-granular assemblies (by default each directory in the web project is compiled into a separate assembly). Note that the web project can also obviously reference other class library projects as part of a solution too (these work just the same as today and would compile into their own project’s assembly).
The big advantage of this more granular model is that the entire web-site does not need to be re-compiled on every change -- instead the scope of compilation can be easily calculated based on what changed, what the down-level build dependencies of the change were, and only the affected pieces need to be re-built.
VS 2005 developers also now have more options in their “Build” menu -- they can force building the entire solution (including the current web project and all other projects in the solution), they can build/re-build just the web project, or they can now also choose to use the new “Build Page” option and build/re-build just the current page they are working on (and any dependent resources like user controls that it is using that have also changed) that they are editing. This provides a great way you can quickly validate local changes you’ve made to a few files without having to-do a full build. For C# developers who don’t have the background compile capability of VB, it also provides a super convenient way to update intellisense without having to-do a full re-build.
The F5 and Ctrl-F5 “run” options can also be configured by the developer to choose what level of compilation error checking should occur when these keystrokes are used to launch a web app. The default continues to be compile all changed pages in the web site – although a useful configuration option when dealing with large web apps is to compile just the pages/resources you are currently working on (this would allow me to open a 9,000 file web-app in 3 seconds, and then compile and run any page within it in about 2 seconds – and avoid the first time hit of compiling everything else).
Deeper and More Accurate Error Checking and Validation at Built-time
Errors found at compile/build time are infinitely easier to identify and fix than ones hit at runtime. One of our goals with VS 2005 and ASP.NET 2.0 was to increase the depth of checking we provide at build-time in VS 2005, and provide a more accurate view of what will happen with the web app at runtime that with VS 2003 built apps.
Today with VS 2003 and ASP.NET V1.1 developers actually use two different build environments (typically without realizing this). One of these is VS 2003 which compiles and builds the code-behind files within the web project using one set of settings (stored in a .proj file) and produces a .dll stored in the \bin directory. The other build-system is ASP.NET V1.1 which at runtime will then use a different build system to parse and compile controls, markup and code within .aspx files that derive from the compiled code-behind classes, and which then saves the output of these compilations into a separate temporary folder on the server. The build-settings of the ASP.NET compilation system, and the subsequent load-time behavior of the code-behind .dll that VS produces, are stored in the application’s root web.config file.
The fact that these two systems are completely different today can be confusing, and can often lead to errors that are missed entirely by VS at compile time – but which cause web apps to blow up at runtime. A few common issues we often hear complaints about:
1) A developer renames the id of a control in the .aspx file, but doesn’t update the field reference in their code-behind. When they build in VS 2003, they don’t see any compile time errors – but at runtime they get a null-reference exception (because there is a mismatch between the .aspx and code behind).
2) A developer has a typo on a server-control definition in the .aspx (wrong attribute, incorrect attribute value, misclosed server control tag, data-binding expression error, template error, etc). When they build in VS 2003, they don’t see any compile errors – but at runtime they get a parse time error.
3) A developer has an error in their web.config file (either bad syntax or an illegal value). When they build in VS 2003, they don’t see any compile errors – but at runtime they get a configuration error.
4) A developer builds a web project. After testing it and thinking it is done, they switch the project from debug->release mode and compile assuming that the deployed project is now running optimized. They forget that the <compilation debug=”true”> section in their web.config is the true switch to control whether the app runs under debug mode, and that VS configures this to be true by default and never changes it. Their app ends up running slowly, and has different behavior (error handling paths are different, .aspx compilation is much slower and inefficient, etc) than it should.
5) Discrepancies between the values in the <assemblies> section in web.config and the references in VS can cause assemblies to be incorrectly loaded or not found.
With VS 2005 and ASP.NET 2.0 we decided to merge the ASP.NET and VS web project build-systems into one common compilation architecture with one shared place to store compilation settings for a web project. Among other benefits, this now means that all of the above 5 issues will now generate compile-time errors when building web projects in VS (both in the IDE and through the command-line with MSBuild) instead of only being caught by testing at runtime.
It also means that we can do much deeper design-time integration inside VS itself – so that things like Master Pages, User Controls, Themes and other resources that have previously been “dynamically compiled” by ASP.NET only at runtime can now be compiled and used within VS as well (for example: this means you now get both WYSIWYG and intellisense support for user-controls, master pages and skin resources). It also allows us to get full intellisense of code+content within .aspx pages – so <script runat=server> blocks, <%# %> databinding blocks, and other syntactical elements inside ASP.NET pages are now fully supported inside the VS editor.
We choose to store the unified compilation and build settings (release/debug, assembly references, namespace imports, warning levels, etc) within the <compilation> section of the application’s web.config (where all of these settings already existed in ASP.NET and V1.1). Because this file is relative to the web project root, it allows the web project to be easily included in multiple .sln solutions within VS.
In addition to providing richer compilation and runtime validation within web projects, we have also added new accessibility compliance validation in VS 2005 (to allow developers to check for Section 508 and WCAG compliance). This accessibility checking can be run manually (there is a button in the toolbar and a menu item to configure and run it), or developers can also configure their web projects to automatically run the accessibility checker each time a web project build occurs.
More Flexible and Iterative Web Development Experience
We’ve tried to optimize the workflow behavior within VS 2005 to enable a more iterative and flexible web editing model. Several of these optimizations are a direct result of the changes we’ve made in the web project system and include:
1) The ability to make quick tactical changes to your web projects in VS 2005 and immediately “save and refresh” to see the changes. With VS 2002 and VS 2003, developers were always required to re-compile and re-build their projects to see the affect of any changes within their web apps. VS 2005 obviously fully supports this as well (as mentioned earlier you can “Build Solution”, “Build Web” and “Build Page” to compile and verify there are no errors at any granularity).
We also then support an “on demand” compilation workflow where a user can just save the code-behind or class file they are editing, and immediately hit refresh in the browser (no explicit re-compile or re-build required). ASP.NET will then on-demand re-compile only the changed resources and apply it to that request – allowing you to immediately see the difference. This is ideal for making quick surgical changes (one or two liners where you are confident you have it right and just want to immediately see the changes). We think this feature merges the best features of dynamic languages/scripting (rapid development) with the best features of strongly-typed compiled languages (better error checking and performance).
2) The ability to both edit and add code while the debugger is attached. With VS 2002 and VS 2003, attaching and detaching the debugger can often be cumbersome and time consuming (especially when it nukes the process on detach). When the debugger is attached, developers are not allowed to make any changes to the running web application or web project.
With VS 2005, we now allow a workflow where developers can launch a web project under the debugger, try out and step through a request to identify an issue, and then – without closing or detaching the debugger – add methods, classes, pages and change code in the project, hit save, and then hit refresh in the browser to immediately step through things in the debugger again. This is supported for both VB and C#, and enables much richer editing support than the “edit and continue” support in client projects (where only code inside methods can be changed – and method, parameter, and class signatures can’t be changed). I personally have found it useful to keep the debugger attached for 1-2 hours at a time, while I work on pages, classes and develop the entire web with the ability to step through things at anytime.
Richer Web Deployment Options
VS 2005 and ASP.NET 2.0 now support several additional deployment options for web projects not previously available with VS 2003. These include:
-- The optional ability to completely compile a web project, including compiling all .aspx, .ascx, and .master file content (allowing you to compile and then remove the HTML and ASP.NET server control markup code, and optionally even remove the .aspx and .ascx files themselves and just ship the .dll assemblies). There are a couple of benefits with this feature: 1) It allows you to ship a totally “locked-down” deployment package for enterprise deployment environments. 2) It allows you to better protect your intellectual property from customers if you don’t want them to see or change your markup. 3) It eliminates all runtime ASP.NET compilation steps, avoiding the first-time performance hit when a page is first accessed and the .aspx markup is compiled, and enabling much better application startup time and working set usage. This later benefit can be particularly noticeable with large applications, and provides a big performance improvement over VS 2003 built apps today.
-- The ability to support “patching” of deployed assemblies within a site, and allowing you to overwrite some (but not all) of the compiled code-behind binaries on your deployment server without having to change all of the DLLs. This is enabled using the “fixed names” option within the new web build-system, and is particularly useful with user control libraries where you might only want to re-issue a binary for one or two controls in a library and not have to update the code-behind dll used for all of them.
Note that in addition to support complete compilation, we also obviously continue to support a model where only the code-behind is compiled and the .aspx markup is left for later customization. We also now support a fully dynamic compilation model where everything (including code-behind and classes) is dynamically compiled on the production server (this is useful for small site and hobbyist scenarios).
Web Project Build Automation Support
VS 2005 and ASP.NET 2.0 now have richer support for command-line level build automation, and can also be integrated and automated using the new MSBuild system being introduced in VS 2005.
ASP.NET 2.0 ships with a new command-line compiler (aspnet_compiler.exe) that provides full support for compiling and building ASP.NET projects (with or without VS installed). It supports all of the new build and deployment options mentioned in web projects earlier (optional compilation of .aspx content, updatable assemblies, etc), and provides deep error warnings and detections when compiling web applications.
The aspnet_compiler.exe utility can be invoked standalone, or used within the context of a MSBuild project file. MSBuild provides a very rich framework for automating and constructing build and environment rules. Some scenarios of how you could use this for web projects: support building and then copying user-control libraries from multiple web-projects into a web-site as a pre-build step, compiling your web projects, code-signing output, producing both release and debug builds in separate directories for each automated build you kick off, kick off automated unit tests, send out code-coverage reports, etc.
MSBuild can be directly run from the command-line (it ships as part of the .NET Framework redist setup – so no VS is required on your build-server). If you pass it a .sln file that has web projects contained within it, these can be built and deployed as part of the solution by MSBuild and provide basic build option support.
Unfortunately we don’t have direct MSBuild IDE support to launch MSBuild workflows in the web project IDE – so you can’t add a MSBuild file into a solution with only a single web project and directly kick it off in the IDE (note: C++ projects also don’t support this either in VS 2005).
What you can do though with VS 2005 web projects to get equivalent functionality is to add a “Build Helper Project” that lives next to your web project in the same VS solution. This “Build Helper Project” is just an empty class library project, but can contain custom build/configured MSBuild scripts where you can specifically script custom build actions for both your web project and overall solution (for example: as part of a single script you could compile multiple user-control library projects, then merge them into a web application project, then compile and publish it, then run a post-grep utility over it to make sure only production deployment connection strings are being used in the web.config file, encrypt any necessary web.config files, etc, etc). You can then kick-off these builds/actions by building the “Build Helper Project” either in the IDE or directly from the command-line. This isn’t as elegant as attaching all of these build scripts to the web project directly, but it enables the same functionality – and provides a lot more power than what was available in VS 2003.
Going forward we are looking to add more integration support with MSBuild in the next VS Orcas release – there is a lot of cool stuff we will be able to-do with it. In the meantime, we are also planning to post some samples on the web of additional MSBuild tasks (w/ source code so you can see and tweak them) that show how to enable both common and advanced scenarios with web projects.
Visual Studio Team System Integration
One of the biggest things in VS 2005 is the introduction of the Visual Studio Team System products, and the advanced development support they provide. These features are fully integrated with the web project system and allow developers building web projects to take advantage of lots of new capabilities. A handful of a few of the big new features that touch a little on the web project system include:
1) Ability to develop unit tests for both classes and web UI within a web project (allowing functional ASP.NET UI validation within test projects). VSTS includes a built-in web replay tool that captures interactions between a browser and the server, and allows easy automation and validation of behavior.
2) Ability to measure and precisely quantify the percentage of code exercised by tests (what we often call code coverage). A web developer can build a web project, then build lots of tests to exercise its functionality, and can then run those tests and get a report detailing that “74% of the lines of code in the web project were executed during the tests”, as well as a precise breakdown of what parts of the code-base were not exercised because boundary and use cases weren’t good enough.
3) Ability to load-test web projects to measure RPS (requests per second) throughput, average latency, memory usage and dozens of other metrics.
4) Ability to profile code execution of web projects, and analyze the % of time spent where within a web project as well as memory allocations and usage.
5) Ability to automate and kick-off regular or nightly builds using the new Build Server capabilities. Tests can automatically be run as part of this to verify and report build quality.
And a whole lot more…
Visit: http://lab.msdn.microsoft.com/teamsystem/default.aspx for more details on Visual Studio Team System.
Sharepoint Server Support
We’ve done a lot of work to integrate ASP.NET 2.0 with the next release of Sharepoint that ships with Office 12 next year. The new version is built entirely using the new feature of ASP.NET 2.0: WebParts, Master Pages, Themes, Site Navigation, Membership, Roles, Personalization, Localization, Data Controls, etc.
One of our goals with Visual Studio is to enable great tool support for building Sharepoint solutions. Today with VS 2003 this is fairly limited (you can pretty much only create a web-part control as a class library). One of the design goals of the new VS 2005 web project system was to enable great Sharepoint support, and enable directly opening and debugging Sharepoint solutions. Going forward we will support creating and editing new pages in running Sharepoint servers, support adding code-behind logic to pages, creating application logic, customizing workflow, building and adding new webpart controls, and debugging the finished solution.
In closing (btw -- thanks for reading this far...)
As the above sections hopefully spell out, there is a heck of a lot of new functionality provided with the VS 2005 web project system. We think this functionality not only adds a lot of new features to take advantage of, but also addresses a lot of the key pain-points we’ve heard feedback from customers using VS 2003 today.
So what are the downsides to it? Well, we really hope there aren’t a lot…. Several people have expressed concerns about some important features missing from the project system in Beta1 and Beta2. As I mentioned in my earlier blog post, though, these features will be in the final release.
The only current VS 2003 feature I know of that is not directly supported in the VS 2005 web project system is the lack of pre/post build events defined directly in the web project itself. As I mentioned above in the “Web Project Build Automation Support” section you can add a “build helper project” into your solution to get back this equivalent functionality, as well as to get much, much richer build customization support than VS 2003 provided using the new MSBuild capabilities. This extra step is a little more inconvenient than embedding directly in the web project, even though it does provide much more power. We will be publishing a whitepaper before the final release that walks through the steps on how to set it up and take advantage of it to help make its usage and benefits more discoverable.
Because there have been a number of changes to both the web project system, and lots of other parts of Visual Studio, it is also going to take a little time for current developers to learn all the new capabilities, how to take full advantage of them, and sometimes how to map and evaluate the new capabilities to existing solutions (and in some cases workarounds) they might be using today to achieve similar results.
It would be great if people could post questions or suggestions here for additional blog entries or MSDN whitepapers that you’d like to see come out between now and the final release that cover either “how do I do [insert scenario]?” or “is there a better way now to handle [insert scenario]?” I’ll then follow-up on these and either post new blog entries on them or get whitepapers written that go into more detail.
Here are a few I’ve already been asked about that we are planning to cover:
1) Best techniques for re-using user control libraries in multiple web apps with VS 2005
2) Best techniques for building a single web-site that is partitioned up into multiple nested/isolated vdirs and applications with VS 2005
3) Best techniques for using MSBuild and builder helper projects with web projects in VS 2005.
4) Best techniques for upgrading from VS 2003 to VS 2005 and what project system considerations you should keep in mind.
Hope this helps provide some more information about the new project system, as well as clear-up some confusion about what it is.
Again – let me know additional things you’d like more details or clarification on.
Thanks,
Scott