See the first post in this series
Tip #2 - How people link to you really does matter
Have you ever taken the time to look at all the different ways you link to your content? For ASP.NET developers there are typically 3 ways to link to the default page (usually default.aspx). For example, let's say there is a landing page in your site for "Products" under the directory /products with a default.aspx page. Therefor you could link to it as:
- example.com/products
- example.com/products/
- example.com/products/default.aspx
Which is the right one to use? This largely depends on what you are doing. My personal preference would be to load up the URL with keywords (more on that in a future post) and to not show the page with the extension.
In my opinion the right choice is:
Why does this matter?
In Google's eyes example.com/products and example.com/products/ (note the trailing slash) are 2 different pages! In fact, example.com/products/default.aspx is considered to be a different page as well!
So let's say you've written a brilliant piece of content and people are linking to it like mad (good for SEO). You're on Digg, cnn.com, and even Mr. Guthrie covered you! Let's say there are 1,000 total links to your post - you are golden right? Well, maybe not.
If 25% of the people linked to example.com/products, 50% linked to example.com/products/, and the remaining 25% linked to example.com/products/default.aspx you would not be maximizing your incoming links. Instead you would be supporting 3 different pages of the exact same content. The number of external incoming links is very important for SEO and if you don't control how people are linking to your content you are reducing your ability to get higher in natural search results.
This can get even worse if you are serving content for 2 domains as discussed here.
How to solve this?
The good news is that solving this is fairly straight forward using HTTP 301 status codes.
A HTTP 301 status code tells the browser to redirect to another URL and that the redirect is permanent. Another status code, 302, is used by ASP.NET developers all the time (using Response.Redirect which we'll cover in a future article) also redirects to another URL but says that the redirect is temporary.
The difference between a 301 and 302 is not noticeable by people browsing a site, but for search engines its meaning is very important: a search crawler will follow a 301 but will not follow a 302.
The recommendation would be for ASP.NET developers to use something like an HttpModule to examine all incoming requests and examine the path of what is being requested. If a request is for /default.aspx or for a page that doesn't have a file extension the HttpModule could short-cut the request and send an HTTP 301 back to the browser redirecting to the right URL:
Response.RedirectLocation = "/products/";
Response.End();
Yes, this also does mean that for every incorrect URL requested you're going to ask the browser to fetch a new URL (2 requests to the server). But the good news is that when search crawlers find the links - no matter the format - they will be given the right link through the 301 redirect and ensuring that you aren't showing different pages for the same content.
What about preventing search engines from accessing certain parts of your site? You can use a robots.txt file as well as a META tag with a noindex option on individual pages. Just be careful about what you include in robots.txt - one of the first places a hacker can look to see what you are trying to hide is your robots.txt file.
Next Tip: Put keywords in the URL