Technical SEO: The Complete Guide

Technical SEO gets described as the foundation, which undersells it slightly. It is closer to a gate. Content quality, keyword targeting, and link building all operate downstream of one question: can a search engine reliably find this page, load it, understand it, and store it? If the answer is no, everything else you do is theoretical.

The good news is that technical SEO is the most tractable part of the discipline. Unlike content quality or authority, most of it is verifiable. A page is either indexed or it is not. A redirect either resolves in one hop or it does not. You can check, fix, and confirm.

This guide walks the pipeline in the order search engines actually experience it, then finishes with a checklist and the tools worth paying for.

How search engines process a website

Four distinct stages, and it helps enormously to keep them separate because the fixes are completely different at each one.

1. Discovery. The engine learns a URL exists, through links, sitemaps, or redirects.
2. Crawling. It requests the URL and receives a response.
3. Rendering. It executes JavaScript and builds the final DOM.
4. Indexing. It evaluates the content and decides whether to store it and what it is about.

Diagnosing which stage is failing saves days. A page that is not in the index might have never been discovered, might be blocked from crawling, might render blank, or might be crawled and rendered fine and then judged not worth indexing. Those are four different problems with four different fixes, and Search Console's Page Indexing report will usually tell you which one you have. The SEO audit guide sets out the full sequence for working through them.

Crawling

Crawl budget is not something most sites need to worry about. If you have fewer than roughly 10,000 URLs, Google will manage. Where it becomes real is on large ecommerce catalogs, sites with faceted navigation, and anything generating URLs combinatorially.

The usual culprits, in rough order of how often we find them:

• Faceted navigation generating millions of filter combinations, each a crawlable URL
• Session IDs or tracking parameters in internal links, creating duplicate URLs for the same page
• Infinite calendar or pagination loops that a crawler will happily follow forever
• Internal links to redirecting URLs, which wastes a request on every crawl
• Slow server responses, since Google throttles crawl rate when your server struggles

Check your raw server logs if you can get them. Log analysis is the only way to see what Googlebot is actually requesting rather than what you think it should be requesting, and the gap is often startling. We have found sites spending the majority of their crawl budget on parameterized URLs nobody knew existed.

Indexing

Getting crawled is not getting indexed. Google crawls plenty of pages and then declines to store them, usually reported as "Crawled, currently not indexed." That message means the page was accessible and Google chose not to keep it, which is a quality and value signal rather than a technical one. Common causes: the page duplicates another page's intent, it is thin, or it sits in a section of the site with no internal links pointing at it. Domain-level authority plays a part too, since Google is measurably stingier about indexing marginal pages on sites with a thin link profile, which is one of the less obvious returns on digital PR.

The levers you control directly:

• <meta name="robots" content="noindex"> keeps a page out of the index. Note that the page must be crawlable for this to be seen, so never combine noindex with a robots.txt block.
• Canonical tags consolidate duplicates onto a preferred URL.
• Internal links signal importance. Orphan pages get indexed slowly or not at all.
• Sitemap inclusion helps discovery but does not guarantee indexing.

A quick audit that catches a surprising number of problems: compare the number of URLs in your sitemap against indexed pages in Search Console. Large gaps in either direction are worth investigating.

Rendering

Google renders JavaScript, but rendering is expensive and deferred. Content that only exists after client-side JavaScript executes will be found eventually rather than immediately, and "eventually" can be weeks on a low authority site.

The practical test takes thirty seconds. Open the URL Inspection tool in Search Console, request live test, and look at the rendered HTML. If your main content, your headings, and your internal links are missing from that output, you have a rendering problem regardless of how the page looks in your browser.

Also compare the raw response with the rendered DOM. In Chrome, view source shows what the server sent, while the Elements panel shows the final DOM. If your title tag, canonical, or primary heading only appears in the second one, move it server side.

Site architecture

Architecture is where technical SEO and strategy meet. The structure of your URLs and internal links tells search engines how your topics relate to each other and which pages you consider most important. We covered the strategic side of this in the SEO strategy guide. The technical requirements are simpler:

• Every indexable page reachable by a crawlable <a href> link, not a JavaScript click handler
• Important pages within three clicks of the homepage
• URL paths that mirror hierarchy, lowercase, hyphenated, no parameters where avoidable
• One canonical URL per piece of content, consistently linked
• No orphan pages, and no pages whose only inbound link is the sitemap

Breadcrumb navigation is worth implementing properly with BreadcrumbList structured data. It reinforces hierarchy and Google frequently uses it in the results display.

Core Web Vitals

Three metrics, all measured at the 75th percentile of real user visits rather than in a lab:

Metric	Good	Measures
Largest Contentful Paint	Under 2.5s	When the main content becomes visible
Interaction to Next Paint	Under 200ms	Responsiveness to user input
Cumulative Layout Shift	Under 0.1	Visual stability during load

Use the Core Web Vitals report in Search Console for field data, and Lighthouse only for debugging. Lab scores are useful for finding causes and useless as targets.

What actually moves LCP, in our experience: serving the hero image in a modern format at the correct dimensions, preloading it, eliminating render-blocking resources in the head, and cutting server response time. What moves CLS: explicit width and height on every image, reserved space for ads and embeds, and self-hosted fonts with font-display: swap. What moves INP: breaking up long JavaScript tasks and deferring third-party scripts that do not need to run on load.

A useful reality check before you spend a sprint here. Core Web Vitals is a real but modest ranking factor. Going from poor to good is worth doing. Going from good to slightly better is usually not the highest value work available to you.

Mobile SEO

Google indexes the mobile version of your site. That is the whole ballgame. If your mobile template hides content behind tabs, omits sections present on desktop, or serves different structured data, the desktop version might as well not exist.

Things to verify: identical primary content on both templates, tap targets at least 48 pixels, no horizontal scrolling at 360 pixels wide, no intrusive interstitials on entry, and identical meta tags and structured data across both.

XML sitemaps

A sitemap is a discovery aid, not an indexing instruction. Keep it clean and it does its job well.

• Only include indexable, canonical, 200-status URLs. No redirects, no noindex pages, no 404s.
• Cap each file at 50,000 URLs or 50MB uncompressed, and use a sitemap index above that.
• Keep lastmod accurate. Google uses it, and inflating it on every deploy trains them to ignore it.
• Skip priority and changefreq. They are ignored.
• Reference the sitemap in robots.txt and submit it in Search Console.

Robots.txt

Robots.txt controls crawling, not indexing. This distinction causes more accidental damage than any other single misunderstanding in technical SEO. A URL blocked in robots.txt can still appear in search results if other pages link to it, because Google knows it exists but cannot see the noindex tag you put on it.

If you want a page out of the index, allow crawling and serve noindex. If you want to conserve crawl budget on pages you do not care about, block them in robots.txt. Never both.

Other things worth checking: never block CSS or JavaScript needed for rendering, keep the file under 500KB, and test changes in the Search Console robots.txt tester before deploying. A misplaced Disallow: / on a Friday afternoon is a genuinely bad weekend.

Canonical tags

Canonicals tell search engines which version of duplicate or near-duplicate content to treat as authoritative. They are a hint rather than a directive, and Google overrides them when it disagrees, which it often does when your signals are inconsistent.

Rules that keep them working: every page gets a self-referencing canonical with an absolute URL. Canonical targets must be indexable and return 200. Do not chain canonicals. Make sure your internal links, sitemap, and canonical all agree on the same URL, since contradicting yourself is how Google ends up picking its own canonical. And do not use canonicals to consolidate genuinely different pages; that is what redirects or consolidation are for.

Redirects

Use 301 for permanent moves and 308 if you need to preserve the request method. Use 302 or 307 only when the move really is temporary, and be aware that Google will eventually treat a long-lived 302 as permanent anyway.

Three habits that avoid most redirect problems. Keep chains to a single hop, since each additional hop adds latency and dilutes signals. Redirect to the closest relevant equivalent rather than dumping everything on the homepage, which Google frequently treats as a soft 404. And when you migrate, update your internal links to point at the new URLs directly instead of relying on the redirects you just built.

Structured data

Structured data does not directly improve rankings. It improves how you appear, and it increasingly affects whether machines can extract facts from your page confidently, which matters a great deal for AI search visibility.

Use JSON-LD. The types worth implementing for most sites: Organization and WebSite sitewide, Article or BlogPosting on editorial content, BreadcrumbList for hierarchy, FAQPage where you have genuine questions and answers on the page, Product with offers and reviews for ecommerce, and LocalBusiness for physical locations.

One hard rule: structured data must describe content actually visible on the page. Marking up FAQs that are not there, or reviews you invented, is a manual action waiting to happen. Validate with the Rich Results Test and Schema.org's validator, which catch different classes of error.

JavaScript SEO

Rendering strategy determines almost everything here.

Approach	SEO reliability	Use for
Static generation	Excellent	Content that changes infrequently
Server-side rendering	Excellent	Personalized or frequently changing pages
Incremental static regeneration	Excellent	Large catalogs with periodic updates
Client-side rendering	Poor	Logged-in areas you do not want indexed

Practical rules for JavaScript-heavy sites: render primary content and internal links server side, use real anchor tags for navigation, never put critical SEO tags in client-only code, and avoid routing that depends on fragment identifiers. Test with JavaScript disabled to see the worst case, then test in the URL Inspection tool to see the realistic case.

Next.js SEO

Next.js gives you good defaults, which makes it easy to assume you are covered when you are not. The specifics that matter:

• Use the Metadata API. Export metadata for static values and generateMetadata for dynamic pages, and set metadataBase so relative URLs resolve correctly.
• Set canonicals explicitly through alternates.canonical. Next.js will not infer them for you.
• Default to server components. Any component marked "use client" that holds your main content moves that content out of the initial HTML.
• Generate sitemap.ts and robots.ts from your actual content source rather than maintaining them by hand.
• Use next/image with explicit dimensions, and set priority on the LCP image only.
• Load third-party scripts through next/script with strategy="afterInteractive" or "lazyOnload".
• Watch trailing slash consistency. Mismatched configuration between Next.js and your host produces avoidable redirect chains.

International SEO

If you serve multiple languages or countries, hreflang is how you tell search engines which version belongs to whom.

Requirements that trip people up: hreflang must be reciprocal, so if English points to German, German must point back to English. Every set needs a self-referencing entry. Use correct codes, so en-GB not en-UK. Include an x-default for your fallback. And hreflang annotations should point at canonical URLs only.

On structure, subdirectories such as /de/ are usually the pragmatic choice because they inherit domain authority and are cheap to maintain. Subdomains and country-code domains both fragment authority, and ccTLDs only make sense when local trust genuinely justifies the cost.

Technical SEO checklist

Work top to bottom. Anything failing in the first block makes the rest academic.

Indexation

• Priority pages return 200 and appear in the index
• No unintended noindex tags in production
• Robots.txt does not block anything important, including assets
• Self-referencing canonicals present and consistent with internal links and sitemap
• Sitemap contains only canonical, indexable URLs and is submitted

Rendering and structure

• Main content and internal links present in server-rendered HTML
• No orphan pages among priority URLs
• Redirect chains resolved to a single hop
• Internal links point to final destination URLs
• Breadcrumbs implemented with structured data

Performance and presentation

• LCP under 2.5s, INP under 200ms, CLS under 0.1 in field data
• Images sized, lazy loaded below the fold, modern formats
• Mobile and desktop serve identical primary content
• Structured data validates and matches visible content
• Hreflang reciprocal and correctly coded, if applicable

Tools

You need less than you think. Search Console is non-negotiable and free, and its Page Indexing and Core Web Vitals reports answer most questions. Screaming Frog for crawling, with the free tier covering 500 URLs. PageSpeed Insights for field and lab performance data together. The Rich Results Test for structured data. Ahrefs or Semrush for site-wide monitoring and competitive context.

Beyond that, server log access is the single most underrated diagnostic available, and it costs nothing but access.

Related reading

• SEO Audit: The Complete Guide for the process that turns these checks into a ranked list of fixes
• SEO Strategy: The Complete Guide for how technical work fits the wider plan
• Content Strategy for SEO for what to publish once the foundations hold
• AI Search Optimization for machine readability and citation
• Digital PR and Authority Building for earning the links and mentions that raise crawl priority
• Local SEO: The Complete Guide for location schema and multi-location structure