One of the great things about getting older is you also become wiser. I'm convinced that I'm not nearly as wise as I should be by this point in my life, but I am still learning some new tricks. To that end, I wanted to cover some improvements that I've implemented in the way my ASP.NET MVC projects address SEO considerations for missing content.
MVC and SEO Redux
As I covered two years ago, one of the greatest benefits of ASP.NET MVC (in my mind) is how it is much more SEO-friendly than Web Forms. More specifically, what I really appreciate about ASP.NET MVC is how much easier it is to enable both "bot-friendly" and "user-friendly" errors when they presented themselves due to either mangled links, changes in URL schemes, or any of the other problems that plague the web in terms of link rot and other problems that can degrade search engine ranking.
Only, with the progression of time, I've been able to see that I was initially using a hammer where I could actually have been using something more like a scalpel. Figuring this out has not only allowed me to create a more streamlined approach to handling missing content on my sites, but it has also made it much easier to add some intelligence into the ways in which I'm processing requests for content that can't be found.
The NotFoundResult in Context
As part of my previous approach to addressing cases where content (notice I didn't say files) was either not found, moved, removed, or moved permanently, I created my own custom ActionResult called a NotFoundResult. This NotFoundResult was then leveraged, in turn, under two primary use cases. The first was in scenarios where a Controller Action was trying to look up an object in the database or elsewhere, and wasn't able to find a corresponding match, as follows:
public ActionResult WidgetDetail(string productId)
{
if(productId.ToLowerInvariant() != "cucumbers")
return new NotFoundResult();
return View("Cucumbers");
}
And the second was by means of setting up a catch-all route that would map to a Controller Action that would, in turn, just throw out a new NotFoundResult of its own:
routes.MapRoute(
"CatchAll",
"{*catchall}",
new { Controller = "Home", Action = "NotFound" }
);
The benefits of this approach were simple: Whenever I needed to report on content that wasn't found (404), had moved (301 or 302), or had been removed (410), I could do so by throwing out the appropriate HTTP Response Codes to allow bots to update indexes accordingly—while still spitting out perfectly "skinned" error pages that were user-friendly and readable.
A Smarter, Efficient NotFoundResult
The problem with this approach, however, was that I didn't need to go as "deep" as implementing a full-blown ActionResult (where I then had to tackle the prospect of wiring up "user-friendly" views on my own). Instead, I later found out that I could much more easily achieve the same results by merely extending an MVC ViewResult. Taking this approach made my code less brittle and tons easier to maintain.
Of course, gracefully handling 404s, 301s, and 410s for both end users and bots is one thing—but being able to proactively address issues with changes to your URL scheme or other perennial problems is another. To that end, I discovered that throwing a bit of logic into the mix to be able to better handle the distinction between, say, a 410 and a 301, was a big win from an SEO perspective.
For example, when making major changes to the URL schemes for some of my sites, I found that the ability to "map" those changes by means of distinctive 301 "handlers" was a great boon. So, in cases where changing a route from something like /products/product/cucumbers/ to /product/cucumbers/, I needed a way to ensure that search engines would pick up on the changes—especially when linked in to my sites from other sites. One way to accomplish this, of course, is to use the "Webmaster tools" section of the search engine you want to notify of the changes—but that's not guaranteed to work. As such, a better way to handle "moved" content is to use HTTP Moved Permanently (301) Response Headers.