ASP.NET 4 SEO Improvements (VS 2010 and .NET 4.0 Series)

[In addition to blogging, I am also now using Twitter for quick updates and to share links. Follow me at: twitter.com/scottgu]

This is the thirteenth in a series of blog posts I’m doing on the upcoming VS 2010 and .NET 4 release.  Today’s post covers some of the improvements being made around Search Engine Optimization (SEO) with ASP.NET 4.

Why SEO?

Search engine optimization (SEO) is important for any publically facing web-site.  A large percentage of traffic to sites now comes from search engines, and improving the search relevancy of your site will lead to more user traffic to your site from search engine queries (which can directly or indirectly increase the revenue you make through your site).

Measuring the SEO of your website with the SEO Toolkit

Last month I blogged about the free SEO Toolkit we’ve shipped that you can use to analyze your site for SEO correctness, and which provides detailed suggestions on any SEO issues it finds. 

I highly recommend downloading and using the tool against any public site you work on.  It makes it easy to spot SEO issues you might have in the site, and pinpoint ways to optimize it further.

ASP.NET 4 SEO Improvements

ASP.NET 4 includes a bunch of new runtime features that can help you to further optimize your site for SEO.  Some of these new features include:

  • New Page.MetaKeywords and Page.MetaDescription properties
  • New URL Routing support for ASP.NET Web Forms
  • New Response.RedirectPermanent() method

Below are details about how you can take advantage of them to further improve your search engine relevancy.

Page.MetaKeywords and Page.MetaDescription properties

One simple recommendation to improve the search relevancy of pages is to make sure you always output relevant “keywords” and “description” <meta> tags within the <head> section of your HTML.  For example:

image

One of the nice improvements with ASP.NET 4 Web Forms is the addition of two new properties to the Page class: MetaKeywords and MetaDescription that make programmatically setting these values within your code-behind classes much easier and cleaner. 

ASP.NET 4’s <head> server control now looks at these values and will use them when outputting the <head> section of pages.  This behavior is particularly useful for scenarios where you are using master-pages within your site – and the <head> section ends up being in a .master file that is separate from the .aspx file that contains the page specific content.  You can now set the new MetaKeywords and MetaDescription properties in the .aspx page and have their values automatically rendered by the <head> control within the master page.

Below is a simple code snippet that demonstrates setting these properties programmatically within a Page_Load() event handler:

image

In addition to setting the Keywords and Description properties programmatically in your code-behind, you can also now declaratively set them within the @Page directive at the top of .aspx pages.  The below snippet demonstrates how to-do this:

image

As you’d probably expect, if you set the values programmatically they will override any values declaratively set in either the <head> section or the via the @Page attribute. 

URL Routing with ASP.NET Web Forms

URL routing was a capability we first introduced with ASP.NET 3.5 SP1, and which is already used within ASP.NET MVC applications to expose clean, SEO-friendly “web 2.0” URLs.  URL routing lets you configure an application to accept request URLs that do not map to physical files. Instead, you can use routing to define URLs that are semantically meaningful to users and that can help with search-engine optimization (SEO).

For example, the URL for a traditional page that displays product categories might look like below:

http://www.mysite.com/products.aspx?category=software

Using the URL routing engine in ASP.NET 4 you can now configure the application to accept the following URL instead to render the same information:

http://www.mysite.com/products/software

With ASP.NET 4.0, URLs like above can now be mapped to both ASP.NET MVC Controller classes, as well as ASP.NET Web Forms based pages.  You can even have a single application that contains both Web Forms and MVC Controllers, and use a single set of routing rules to map URLs between them.

Please read my previous URL Routing with ASP.NET 4 Web Forms blog post to learn more about how the new URL Routing features in ASP.NET 4 support Web Forms based pages.

Response.RedirectPermanent() Method

It is pretty common within web applications to move pages and other content around over time, which can lead to an accumulation of stale links in search engines.

In ASP.NET, developers have often handled requests to old URLs by using the Response.Redirect() method to programmatically forward a request to the new URL.  However, what many developers don’t realize is that the Response.Redirect() method issues an HTTP 302 Found (temporary redirect) response, which results in an extra HTTP round trip when users attempt to access the old URLs.  Search engines typically will not follow across multiple redirection hops – which means using a temporary redirect can negatively impact your page ranking.  You can use the SEO Toolkit to identify places within a site where you might have this issue.

ASP.NET 4 introduces a new Response.RedirectPermanent(string url) helper method that can be used to perform a redirect using an HTTP 301 (moved permanently) response.  This will cause search engines and other user agents that recognize permanent redirects to store and use the new URL that is associated with the content.  This will enable your content to be indexed and your search engine page ranking to improve.

Below is an example of using the new Response.RedirectPermanent() method to redirect to a specific URL:

image

ASP.NET 4 also introduces new Response.RedirectToRoute(string routeName) and Response.RedirectToRoutePermanent(string routeName) helper methods that can be used to redirect users using either a temporary or permanent redirect using the URL routing engine.  The code snippets below demonstrate how to issue temporary and permanent redirects to named routes (that take a category parameter) registered with the URL routing system.

image

You can use the above routes and methods for both ASP.NET Web Forms and ASP.NET MVC based URLs.

Summary

ASP.NET 4 includes a bunch of feature improvements that make it easier to build public facing sites that have great SEO.  When combined with the SEO Toolkit, you should be able to use these features to increase user traffic to your site – and hopefully increase the direct or indirect revenue you make from them.

Hope this helps,

Scott

35 Comments

  • great news about MetaKeywords and MetaDescription attributes but why not to allow a KeyValueCollection which contains all available meta tags? it would make it even easier if i would like to add some customized meta tags.

  • great info,keep it up
    thanks

  • Hi Scott,

    Thanks for your great articles. I'm currently trying to debug/solve a problem where the start-up time of our web application has increased to 20-30 seconds. Hence, the "Auto-Start ASP.NET Applications" article was quite interesting - unfortunately, comments are closed on that article. The problem is we have multiple installations per server and thus would like to avoid the extended memory-use by keeping all the applications alive all the time. From one of your comment answers I can see that this could be a problem with the auto-start feature.

    I've been working on using the precompilation feature (the task corresponding to aspnet_compiler.exe in msbuild) in order to reduce the startup time, but this seem to be impossible to apply to our scenario with multiple asp.net web application projects all being compiled seperately and then copied together in the end. Several of the *.compiled files from different projects has name clashes and the references to "/filename.aspx" inside doesn't hold true after copying the files, so a no-go in our situation I'm afraid.

    Anyway, as I understand it, the basic problem is that with a basic asp.net web application, the aspx/ascx files are compiled at the server once the site is visited the first time after startup/worker process recycle. You can then
    a) do nothing - and wait the first time
    b) precompile the website and thereby take care of this task once and for all
    c) use a "poor mans solution" and call a keep-alive webpage in the application every x minutes (costs memory)
    d) use the auto-start feature of asp.net 4.0 (costs memory)

    Why isnt there a b-solution where you instruct the IIS to do the pre-compilation and cache the results. Much like solution b), but without the problems (deployment-wise from my point of view) of manually running the aspnet_compiler.exe. Or is there a such solution? Some way to take a deployed site in the IIS and "do whatever the IIS _would_ do". Maybe what I really miss is a description of what actually happens at that time, and how I can help speed that up or detect exactly which parts of it that are time consuming. Are there tools to assist in debugging/measuring the startup process?

    I hope you have time to answer (some of) my questions :o)

    Regards,

    Jan Hansen

  • Great article! Thanks

  • Nice as always. It would also be useful to have a property to set the canonical URL for a page, although it's not hard to add it manually.

  • Great article and the new IIS SEO kit is really helpful. Heads-up: meta-keywords are no longer used by search engines for ranking purposes; personally, I still put them in to "future-proof" myself in case the SE people change their minds again :-)

  • Great! SEO improvements always helps in creating a structured web site from search engines perspectives.

  • This is great, My clients tend to want to continually change and tweak the meta tags, page titles, and urls. With these tools I have been able to (beta) a new system where the user can go into a web form select the page, update the meta tags, and "re-route" / key in a user friendly URL. So far I have this has allowed me to auto mate the updating of the meta tags via database calls which I cache, and the web form sends me an email to notify me of the re-route. I am working on automating that part as well. But in the soon future I hope to put all of that into the business leaders hands and off my stack of to do. When you have as many projects as me, the less updating of "minor" tasks means more time for improving projects. For example I had to add a keyword to one project a while back that had 200 pages, and 25 master pages. Needless to say wasn't happy that they wanted to add a keyword. This allows me to just step away. Since I have little say in the decision.

  • A couple of questions:
    Am I correct in assuming that the @Page Keywords="" directive is just as valid in MVC as it is in WebForms?

    when SEO algorithms change, does the SEO toolkit inform us that we ought to run it over the sites we have already optimized again to see if anything new has to be updated?

    Many thanks.

  • Jan, you can precompile. There are thousands of such articles on Google.

  • Peter, yes I can precompile. But try to create two "plugin" projects and a "master" project. Each plugin project with "control.ascx". This will end with two control.ascx.cdcab7d2.compiled and two App_Web_control.ascx.cdcab7d2.dll files in each /bin folder. How do you merge this into the /bin folder of the "master" web site? Also, each *.compiled file assumes that the corresponding *.ascx/*.aspx file is placed in the root of the application, which isn't the case after I deploy the plugins into a /plugins// folder structure.

    Currently this is our setup as we want to be able to support third-party development of plugins which it should be possible to just "drop in" to the master application which can then load the plugins.

    So yes, I can precompile all projects individually but I cant see how I can merge all output into one folder structure. Any help, ideas or pointers?

    Basically - what I need to understand is based on this:

    1) you compile (from source!) an asp.net web application into aspx and dll files. Upon first request, IIS precompiles and stores the output where?

    or

    2) you precompile an asp.net web application into placeholder aspx files, .compiled and .dll files. Upon first request, IIS just spits out the content, as precompilation is taken care of.

    Why can't I get a mix of this? I want to compile and deploy my aspx and dll files to the IIS, and then (kindly...:o) ask it to "do whatever it would do anyway" in terms of precompilation and store it on the disk so that this would not be done when the first visitor comes by. Apparently, it can do it (as the site works) so it isn't technically impossible - it just isn't persisted on disk. Also, it is possible to do it beforehand with aspnet_compiler for projects structured in a simpler way than our setup. I don't need it to be "up and running" with the auto-start feature so that first request takes < 100ms , I just want to avoid the precompilation punishment.

    Sigh... I feel like I've read most of the mentioned thousands articles and I havent got any deeper into exactly what happens during the initial wait and what I can to to avoid it except from precompile (doesn't work for me), or keepalive.aspx (which costs memory). Scott - can I buy your time to help look into this problem?

    Regards, Jan

  • I just installed Visual Web Developer Express 2010 at work. I haven't used an express edition in a while. When I created a new project it has this "Account" folder that registers users. Where can I find out about this? How does it work? Where's the database that keeps the user accounts (and how can it be changed)? What's with the second web.config file?

    Thanks

  • Great news!

    The Response.RedirectPermanent() is a great addition and will be welcomed by many of us on webhosts who require us to redirect the non-www address to the www domain address.

    How about some more cool posts on SilverLight 4 Scott? Bring em on!

  • RedirectPermanent is the feature I really like most in ASPNET 4. Nice stuff Scott

  • @Roni,

    >>>>>>>> great news about MetaKeywords and MetaDescription attributes but why not to allow a KeyValueCollection which contains all available meta tags? it would make it even easier if i would like to add some customized meta tags.

    You can actually do this today (although not quite the same way you proposed). Here is a blog post that describes how: http://ryanfarley.com/blog/archive/2006/03/25/18992.aspx

    Thanks,

    Scott

  • @Jan,

    If you can send me email with your details I can loop you in with some folks who can help suggest some things to investigate to improve the startup time.

    Thanks,

    Scott

  • @LeeC,

    >>>>>> Good stuff, Scott. I think you should always mention the IIS URL Rewriter whenever you promote URL Routing.

    Agreed - the IIS URL Rewriter is indeed great!

    Thanks,

    Scott

  • @Alverant

    >>>>>>> I just installed Visual Web Developer Express 2010 at work. I haven't used an express edition in a while. When I created a new project it has this "Account" folder that registers users. Where can I find out about this? How does it work? Where's the database that keeps the user accounts (and how can it be changed)? What's with the second web.config file?

    The pages within the Account folder use ASP.NET's Forms Authentication support to implement a login form. It then uses the ASP.NET Membership system to store users/passwords. By default it stores these values within a SQL Express database.

    If you want a blank project you can create one by choosing the "Empty ASP.NET Web Application" project template.

    Hope this helps,

    Scott

  • great improvement, the SEO toolkit really quite helpful

  • Nice features, thanks for great job you and your team have done!

  • nice article and thanks for metakeyword and metadescription..

  • Ok. it was very good article to new beginers to ASP.NET 4.0.

  • Cool new Features. I really can't wait for asp.net web forms.

  • Loving the series on 4.0. It seems a little odd to me that the team decided to go with Response.RedirectPermanent() instead of a new overload of Response.Redirect() where you can specify the status code to use with a flag or enum. Since the method is essentially doing the same job as Response.Redirect except changing the status code in the response header. A moot point, just curious

  • Thanks for the update, it is always appreciated. To be honest however, I yawn with web technology, WPF is what I find fascinating.

  • Great quality blog, I have always had a great interest in ASP and you have fuelled that interest with knowledge also.

    Thank you

    Mark McCulloch

  • "Response.RedirectPermanent() Method" and "URL Routing" are nice features. By other side meta keywords and meta descriptions don't give any weight for better page rainking. Convinient for fast settig, but useless for SEO.

  • Great work

  • Hi it was realy great nd fresh information.I used to code in .net but I never used this tags.So now I can also create title and meta tags of my dynamic pages with this.

  • Thats an excellent pice of article by you sir !

    Thanks,
    Thani

  • Cool new features!

    One question - is there any reason that ViewState exists on top of the page. Can it be moved to bottom of the page. I believe that should improve SEO rankings.

    Thanks

  • Scott,

    Thanks but it doesn't really help. I haven't heard about any of those things before. I've just been using the traditional blank solution and doing everything by hand. I'm trying to learn how things work now, but I haven't been able to find anything that explains the account folder and the aspnetdb.mdf file.

    "The pages within the Account folder use ASP.NET's Forms Authentication support to implement a login form. It then uses the ASP.NET Membership system to store users/passwords. By default it stores these values within a SQL Express database."

    If it uses an SQL Express database, then why can't I open the aspnetdb.mdf file using SQL Server Management Studio? What are "Forms Authentication" and "ASP.NET Membership system"?

  • Wow! i agree! i’ve been searching for so long for a site where i could find everything that i want, and i’ve just found it!! really, i’ve visited your blog, and it’s amazing, i will keep visiting

  • This is Regarding Response.RedirectPermanent() Method
    ---------------------------------------------
    above it was mentioned that Response.Redirect issues HTTP 302 Found, but as per my knowledge Server.Transfer won't issue 302. So RedirectPermanent is replace ment for Redirect where we can not choose Server.Transfer.

    Please correct me if I am wrong.

  • @duttavr: Server.Transfer won't change the url displayed on the users browser so one woudn't know the page has moved.

    Great features, much anticipated!

Comments have been disabled for this content.