Sitecore: Returning a 404 response on the page requested rather than redirecting to a 404 page

Previously I’ve blogged about:

but while looking through the posts today, I realised I had never written about how you stop Sitecore from issuing 302 redirects to your 404 page and instead return a 404 on the URL requested with the contents of the 404 page.

While search engines will recognise a 302 response to a 404 as a 404 (in fact they’re intelligent enough to work out that a 404 page without a correct response status code is a 404) it’s considered SEO best practice for the URL to stay the same and to issue the correct status code.

Creating a NotFoundResolver class

When Sitecore processes a request it will run the httpRequestBegin pipeline, and within that pipeline is a Item Resolver processor that will attempt to find the requested item. If after this the context item is still null then the logic to redirect to the ItemNotFoundUrl will kick in. To stop this happening we can simply add our own process to the pipeline after ItemResolver and set the item.

Our class looks like this:

using Sitecore;
using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Diagnostics;
using Sitecore.Pipelines.HttpRequest;

namespace Pipelines.HttpRequest
{
    public class NotFoundResolver : HttpRequestProcessor
    {
        private static readonly string PageNotFoundID = Settings.GetSetting("PageNotFound");

        public override void Process(HttpRequestArgs args)
        {
            Assert.ArgumentNotNull(args, nameof(args));

            if ((Context.Item != null) || (Context.Database == null))
                return;

            if (args.Url.FilePath.StartsWith("/~/"))
                return;

            var notFoundPage = Context.Database.GetItem(new ID(PageNotFoundID));
            if (notFoundPage == null)
                return;

            args.ProcessorItem = notFoundPage;
            Context.Item = notFoundPage;
        }
    }
}

To add our process to the pipeline we can use a patch file like this:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <httpRequestBegin>
        <processor
          patch:after="processor[@type='Sitecore.Pipelines.HttpRequest.ItemResolver, Sitecore.Kernel']"
          type="LabSitecore.Core.Pipelines.NotFoundResolver, LabSitecore.Core"  />
      </httpRequestBegin>
    </pipelines>
    <settings>
      <!-- Page Not Found Item Id -->
      <setting name="PageNotFound" value="ID OF YOUR 404 PAGE" />
    </settings>
  </sitecore>
</configuration>

Notice the setting for the ID of your 404 page to be loaded as the content.

Remember if you do do this make sure you also follow one of the methods to return a 404 status code, otherwise you will have just made every URL a valid 200 response on your site.

Updating the response headers on your 404 Page in Sitecore

A few weeks ago I blogged about how to create a custom 404 Page in Sitecore. Following on from that, one thing you may notice in the response header of your 404 Page is the status code is 200 Ok, rather than 404 Page not found.

When Sitecore can’t find a page what actually happens is a 302 redirect is issued to the page not found page, which as its an ordinary page will return a 200 Ok. Thankfully Google is actually quite good at detecting pages a being 404’s even when they return the wrong status code, but it would be better if our sites issues the correct headers.

Method 1

The simplest solution is to create a view rendering with the following logic and place it somewhere on your page not found page. This will update the response headers with the correct values.

@{
      Response.TrySkipIisCustomErrors = true;
      Response.StatusCode = 404;
      Response.StatusDescription = "Page not found";
}

However personally I don’t think this a particularly neat solution. The contents of a view should really be left for what’s going in a page rather than interfering with its headers, even if it does have access to the Response object.

Method 2

Rather than using a view my solution is to add some code to the httpRequestEnd pipeline that will check the context items Id against a setting where we will store the Id of the 404 page item in Sitecore and if the two match then update the response header.

The solution will look like this

Pipeline logic

using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Pipelines.HttpRequest;

namespace Pipelines.HttpRequest
{
    public class PageNotFoundResponseHeader : HttpRequestProcessor
    {
        private static readonly string PageNotFoundID = Settings.GetSetting("PageNotFound");

        public override void Process(HttpRequestArgs args)
        {
            if (Sitecore.Context.Item != null && Sitecore.Context.Item.ID == new ID(PageNotFoundID))
            {
                args.Context.Response.TrySkipIisCustomErrors = true;
                args.Context.Response.StatusCode = 404;
                args.Context.Response.StatusDescription = "Page not found";
            }
        }
    }
}

Patch config file

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>    
    <pipelines>
      <httpRequestEnd>
        <processor
          patch:after="processor[@type='Sitecore.Pipelines.PreprocessRequest.CheckIgnoreFlag, Sitecore.Kernel']"
          type="Pipelines.HttpRequest.PageNotFoundResponseHeader, MyProjectName"  />
      </httpRequestEnd> 
    </pipelines>
    <settings>
      <!-- Page Not Found Item Id -->
      <setting name="PageNotFound" value="ID of 404 Page" />
    </settings>
  </sitecore>   
</configuration>

What’s the TrySkipIisCustomErrors property

Quite simply this stops a scenario where you end up on IIS’s 404 page rather than your own. If you don’t set this, when you update the header status code to 404, IIS likes to return the page from it’s settings rather than continuing with your own.