Deadweight

I’ve touched on the topic of deadweight a few times in this book, and that’s for a good reason. Deadweight pages, that is, any page that is not a targeted page or the start page, should either

• not exist
• not be indexed
• not be systematically linked
• not listed in the Sitemap.xml

However, there are some special cases where deadweight pages have a reason to exist, for instance search pages. User input triggered search pages have no QC, as the user can search for anything, meaning potentially any page might get created and might be indexed. Although search pages are deadweight, they have a user case, and it makes sense that search pages have a shareable URL (so that users can bookmark and share their searches).

To deal with these pages, simply set meta robots to noindex <meta name="robots" content="noindex"> in the HTML head. You also shouldn’t systematically link to deadweight pages, as you do not ever want them to be crawled, especially if there are many or a growing number of them.

Another good example of deadweight pages that may exist is “pagination pages”. Pagination pages do not have a unique dominant targeted phrase because the targeted phrase of the paginated page is always the same as the first page. As there is a reason that paginated pages exist, set them to <meta name="robots" content="noindex">. Additionally, on big sites with hundred of thousands to millions of pages I recommend to not even link to paginated pages systematically. Use some JS magic without direct links to lead the user through the pagination.

You should also know it is possible to set the canonical of a paginated page to the first (unpaginated) targeted list page, so for example, https://www.example.com/c/seo?page=7 -> <link rel="canonical" href="https://www.example.com/c/seo">, as a paginated page is just variation of the unpaginated page. For small site settings, the paginated pages to noindex is fine, setting the canonical to the unpaginated URL is just necessary on really big sites. (Huge sites should best not link to paginated pages at all, use JS pagination instead).

It’s also imperative that Google can discern between pages that exist and pages that don’t. Whenever Google encounters a HTTP 200 page that has no content or has an error message, this is called a “soft 404 error”. Errors have a serious impact on crawling and indexing, and if widespread, can greatly effect your whole site’s traffic performance.

You are also seriously hurting your crawling when you always redirect away from a page that does not exist, for example, https://www.example.com/hudriwudri -> HTTP 301 -> https://www.example.com/file-not-found -> HTTP 404. If a site does not exist, you must return a HTTP 404 directly, without dragging Google (and your user) over a chain of redirects.

However, if you move a page from one URL to another URL, then HTTP 301 Permanent Redirect is correct (for example, https://www.example.com/hudriwudri -> HTTP 301 https://www.example.com/a/hudriwudri-new).

If you have deleted a page that previously existed, you should return a HTTP 410 Gone. This has two implications:

• Google will get rid of the URL in its index faster than with a HTTP 404
• you can make a distinction between real errors (404) and wanted deletions (HTTP 410) in GSC

Having HTTP 404 occasionally on your pages is normal, and will not hurt your overall website’s performance. However, if they are growing in numbers, you should “fight” them and fix the source of these errors. To identify real errors, you must be able to make a distinction between the HTTP 404 and the wanted errors HTTP 410. On any halfway big, or halfway complex website, this distinction is a must.

If you delete parts of your website (HTTP 410), then the remaining parts of your website will not suddenly perform worse. You will lose the value of the inlinks to the deleted pages and, of course, the traffic that reached those pages will be gone, but the rest of your site will keep performing.

Don’t be afraid to delete pages. Do not keep old pages, or pages that you do not need or want to maintain any longer, just because of SEO. If your site becomes unmanageable and unmaintainable because of the old stuff, then no search engine optimization will save you.

Edge Cases

There are some edge cases in regards to deadweight, namely pages for which you do not particularly care if they are found, but nonetheless are necessary for branded go-queries of your users, such as: [%your brand% login], [%your brand% contact]. Therefore, they must exist and be indexed. The pages I’m talking about are pages like:

• mandatory legal information, such as imprint
• contact details
• terms of service
• login
• other common service pages for your vertical.

These pages might not be optimized in the common sense, but nevertheless, should always:

• fulfill the URL rules
• be indexable (are not set to noindex)
• have an optimized title and headline
• be linked from the startpage and/or sitewide (normally via the footer)

Although these are not targeted pages, the page structure and content should still indicate what it contains, contributing to a good user experience. If you have less than a handful of these pages, do not worry if they are in a sitemap or not. If you have more than a handful, I recommend submitting them via the sitemap.