Technical SEO in 2026: How We Audit Sites So Google and AI Search Can Crawl, Index and Cite Them

Technical SEO is the foundation that decides whether your website gets found or buried, no matter how good your content is. Google, and the AI platforms now reshaping how people discover information, can only surface your pages if they can crawl, render and index them without friction. When we open a new audit, the first thing we are looking for is not keywords or backlinks. It is whether the machines can actually read the site at all, because if they cannot, nothing else you do will land. This guide covers every major component of technical SEO, how each one connects to AI search visibility in 2026, and what a proper technical audit looks like when the goal is revenue, not a tidy report.

What Technical SEO Actually Means

Technical SEO is the work of improving your website's infrastructure so search engines can crawl, index and render your pages properly, which is what lets your content compete in both Google Search and AI search experiences. It covers the essentials that sit underneath your content: site speed, mobile usability, internal architecture, structured data, canonicals and crawl controls. Each one affects whether a search engine or an AI system can access and understand what you have published.

That matters commercially. Google uses the mobile version of your content for indexing and ranking, and slow, hard-to-crawl pages quietly cost you revenue before a single ranking factor comes into play. The job of technical SEO is to remove that friction so the rest of your marketing can work.

From Technical Fixes to Revenue Growth

Strong technical foundations are only half the equation. In our SaaS SEO case study, a B2B legal-tech company went from technical issues limiting its growth to generating $1.31M in attributed revenue with a 1,909% ROI in 12 months, by combining technical fixes with an AI SEO strategy. If you want every fix tied to a measurable outcome rather than a crawler severity score, our technical SEO consulting maps each finding to a revenue opportunity. You can also book a 45-minute Discovery Session with us to see how it fits a full campaign.

Why Technical SEO Matters More Than Ever in 2026

Technical SEO has always mattered, but its importance has sharpened in 2026 for two connected reasons. Google is placing more weight on user-experience signals like page speed, interactivity and mobile usability. And AI search interfaces are changing how content gets discovered, with several third-party platforms introducing crawler-specific considerations of their own.

There is a sequencing lesson in this that we now give every client. Address technical SEO before you invest heavily in content production. The technical foundation determines whether AI systems can access, understand and cite your content at all, so pouring budget into articles on a site that AI crawlers cannot read is spending on a leak. Get the plumbing right first, then scale the content.

The stakes are rising because AI has become a real referral channel. In the campaigns we run, AI-referred sessions are growing quickly year on year, and the visitors arriving from platforms like ChatGPT often come with stronger buying intent than a typical organic visitor. Ignoring AI search means leaving that demand on the table.

Google's Page Experience Signals

Google's Core Web Vitals are part of its page experience signals, so page speed, visual stability and interactivity all feed into how your pages perform. Strong scores support a better experience and can help search performance, but Google is clear that good Core Web Vitals scores alone do not guarantee top rankings. Treat them as an important technical foundation, not a shortcut to position one.

Mobile-first indexing raises the stakes further. Because Google now uses the mobile version of your content to index and rank your site, a strong desktop design will not rescue you if key content, usability or performance falls short on mobile. Load speed is especially unforgiving. Google's research found that as a mobile page's load time goes from one second to three seconds, the probability of a visitor bouncing rises by 32%, and it keeps climbing from there. That is revenue at risk from technical neglect, not a vanity metric.

AI Search Platforms Have Their Own Technical Conditions

The rise of ChatGPT, Perplexity, Google AI Overviews and Microsoft Copilot has added a new technical dimension. The rules vary by platform. For Google AI Overviews and AI Mode, Google says there are no additional technical requirements beyond normal Search and snippet eligibility. For third-party platforms like ChatGPT and Perplexity, crawler access and clean HTML accessibility directly affect whether your content can be discovered and cited.

In practice, technical SEO for AI search means building a site that machines can access, interpret and retrieve at scale. That includes clean, crawlable URLs, accurate XML sitemaps, accessible HTML content, structured data that clarifies meaning, and allowing the relevant AI crawlers where appropriate. If your content is blocked, buried behind weak technical implementation, or hard to parse, it is far less likely to be surfaced or cited in AI-driven answers. For a deeper look at the retrieval pipeline behind these answers, our guide to how AI search works walks through it step by step.

The Core Components of Technical SEO

Technical SEO covers a wide range of elements. Here is what each core component does and why it matters.

Site Architecture and URL Structure

Your site's architecture is the starting point. A logical, hierarchical structure helps bots crawl efficiently, distributes link authority to important pages, and makes navigation intuitive for people. A good structure uses clean, descriptive URLs with hyphens separating words, keeps important pages reachable within about three clicks of the homepage, and uses breadcrumb navigation to reinforce hierarchy.

Internal linking works hand in hand with architecture. Use descriptive anchor text instead of generic phrases like "click here", and link deliberately to your key service pages and pillar content so authority flows to the pages that need to rank. Our keyword mapping guide explains how to align architecture with your target keywords across a content cluster.

Crawlability and Robots.txt

Before a search engine can index your pages, it has to crawl them. Crawlability issues stop bots reaching content you want indexed, while accidentally open pages let bots waste crawl budget on content that adds no value. Your robots.txt file controls what bots can and cannot access. The common mistakes we see are blocking JavaScript or CSS files that search engines need to render pages, and blocking entire subfolders that contain valuable content.

For AI search, the same principle applies with an extra layer: review crawler access deliberately, by name. The major AI crawlers you may want to allow include:

• GPTBot and OAI-SearchBot: OpenAI's training and search crawlers; OAI-SearchBot is the one tied to ChatGPT search visibility.
• ChatGPT-User: the agent that fetches a page when a ChatGPT user follows a link.
• PerplexityBot: Perplexity's crawler for search visibility on that platform.
• ClaudeBot: Anthropic's crawler.
• Google-Extended: Google's separate control for whether your content is used to improve its generative models, kept distinct from standard Search indexing.

Google AI Overviews and AI Mode do not need a separate AI crawler allowance. Pages simply need to be indexed and eligible to appear with a snippet in Google Search.

XML Sitemaps and Indexing

An XML sitemap is a directory of the pages you want search engines to find and index. Submitting an updated sitemap through Google Search Console signals which pages matter and keeps the index aligned with structural changes. Effective sitemaps include only canonical URLs, exclude noindex pages, and use accurate lastmod timestamps when content has materially changed. Indexing is a separate problem from crawling: a page can be crawled but not indexed if it is set to noindex, returns an error, or is treated as duplicate. Google Search Console's Page Indexing report is the fastest way to find pages that are crawled but not indexed, and why.

Core Web Vitals and Page Speed

Core Web Vitals measure three dimensions of experience: Largest Contentful Paint (LCP), how fast the main content loads, with a threshold under 2.5 seconds; Interaction to Next Paint (INP), how responsive the page is, targeting under 200ms; and Cumulative Layout Shift (CLS), how visually stable the page is as it loads, targeting under 0.1.

Rather than list every generic speed tip, here are the fixes that actually moved the numbers in a recent B2C client audit, in roughly the order of impact: we fixed duplicate meta descriptions, implemented a single optimised H1 on every page, reduced unused CSS by limiting Google font weights and families, compressed and converted images to WebP, added explicit width and height attributes to prevent layout shift, and added descriptive alt text across all images. By October 2024 that site held a 98/100 technical health score, and we maintained it across the campaign. Core Web Vitals is a deep topic in its own right; our Core Web Vitals audit guide covers the four-step method and the measurement detail.

Mobile-First Optimisation

Since Google moved to mobile-first indexing, the mobile version of your site is what determines how you are evaluated and ranked, so your mobile pages need to expose the same core content, metadata and structured data as desktop. Responsive design is the baseline, not the finish line. It means tap targets large enough to use on a touchscreen, no intrusive interstitials blocking content, navigation tested for mobile rather than just desktop, and real-device testing rather than emulator checks. Rankings frequently diverge between devices, which is why device-specific rank tracking on mobile and desktop is worth the effort. For diagnosis, lean on Lighthouse, PageSpeed Insights and Search Console's Core Web Vitals reporting.

HTTPS and Site Security

HTTPS is a confirmed Google ranking signal and a baseline trust requirement. Sites still on HTTP are flagged as insecure by browsers, which lifts bounce rates and signals poor trustworthiness. Migrating means obtaining an SSL/TLS certificate and updating all internal links from http to https. Security also protects crawl budget: blocking login pages, admin panels, cart URLs and other state-changing URLs keeps crawl resources focused on pages that should rank. Treat HTTPS as essential hygiene, but avoid claiming it directly increases AI citation likelihood unless you have platform-specific evidence.

Structured Data and Schema Markup

Structured data is markup added to your code that helps search engines, and AI platforms, understand what your content means, not just what it says. It is implemented in JSON-LD using Schema.org vocabulary, and it can make eligible pages appear with enhanced features like review snippets and product details. For AI search it provides extra context about a page's meaning and supports cleaner extraction of key information, though it does not guarantee citations on its own.

Where most sites go wrong is treating schema as a box-ticking exercise rather than a system. Our approach is built around two principles: entity consistency and tiered implementation.

• Tier 1, core business pages: your homepage, service pages and contact page carry full Organisation, LocalBusiness, Service and Person detail.
• Tier 2, content pages: blog posts, case studies, category and resource pages reference the Organisation by its @id rather than repeating every detail. This keeps the markup lightweight and consistent across the site.

Every implementation we build uses the @graph format, which groups multiple entity types in a single JSON-LD block and lets entities cross-reference each other through @id values without duplicating information. The most impactful schema types for most sites are FAQPage, HowTo, Organisation, BreadcrumbList and Article. Our schema SEO guide sets out the full methodology.

Canonicalisation and Duplicate Content

Duplicate content forces search engines to choose which version of a page to rank, and they often choose wrong. Canonical tags tell them which version is definitive and consolidate ranking signals to a single URL. The usual culprits are HTTP and HTTPS versions both accessible, www and non-www both indexed, trailing-slash variants, pagination without proper canonical handling, and print-friendly versions exposed to bots. Canonicals are not the only tool: use 301 redirects when one URL permanently replaces another, and noindex for pages that serve a user purpose but should not rank, such as internal search results or filtered category views.

JavaScript Rendering

JavaScript is a growing challenge. Many modern sites use frameworks like React, Angular or Vue to deliver content dynamically, but not every crawler handles client-side rendering the same way. Googlebot does render JavaScript, but with delays, so Google may index an older or incomplete version of a page. AI crawlers are generally less patient than Googlebot and often will not execute heavy JavaScript at all, so content that only appears after a script runs can be skipped entirely. Server-side rendering or static generation are the cleanest solutions. Where those are not feasible, the principle is progressive enhancement: deliver core content and structured data in the base HTML, then layer JavaScript on top.

What a Technical SEO Audit Actually Covers

A technical SEO audit is a systematic analysis of your site's infrastructure to find the issues limiting crawlability, indexation, performance and visibility. For established businesses, run one at least once a year, after any major migration, and whenever organic traffic drops unexpectedly.

Before listing the components, it helps to name why most technical audits fail to move anything. We see the same three failure modes again and again:

1. Volume over value: a 300-issue export with no prioritisation, so the team fixes whatever is easiest rather than whatever matters.
2. Technical SEO in isolation: technical fixes disconnected from content and commercial goals, so nothing connects to revenue.
3. Ignoring AI search: no AI visibility check at all, which leaves one of the fastest-growing search channels completely unmeasured.

A thorough audit that avoids those traps covers:

• Crawl analysis: mapping crawl depth and finding orphan pages, broken internal links, redirect chains and robots.txt blocks, using a crawler like Screaming Frog or a site audit tool.
• Index coverage review: using Google Search Console to find pages excluded from the index, soft 404s and crawl anomalies.
• Core Web Vitals assessment: identifying performance gaps by device and page type.
• Mobile UX audit: reviewing key templates for mobile layout, usability and performance.
• Structured data validation: confirming schema is correctly implemented and error-free, then exporting structured-data reports from every priority page type and building a gap analysis that drives the implementation roadmap.
• Redirect and canonical audit: mapping redirect chains, loops and canonical inconsistencies.
• Security check: confirming HTTPS is correctly implemented across all pages and resources.
• AI crawl accessibility: verifying that relevant AI crawlers such as OAI-SearchBot, PerplexityBot and ClaudeBot are allowed in robots.txt and that core content sits in the base HTML.
• AI visibility spot-check: querying key terms in ChatGPT, Perplexity and Google AI Overviews to see which sources get cited. If competitors appear in AI answers for your target terms and you do not, that is a signal to review both technical and content factors.

What separates a useful audit from a report that gathers dust is revenue-based prioritisation: ranking fixes by likely business impact rather than raw technical severity. That is the difference between a list and a plan. Our technical SEO audit guide goes deeper on the method.

Here is what that looks like in practice. When we took on a B2B property management client, the site started from a 61 technical health score with 1,030 issues, including 229 critical errors. We worked through broken links, image compression, HTTPS and HTTP conflicts and orphan pages, and by month three the site reached a 99 health score. Paired with AI-optimised content, that technical work supported $5.9M in attributed revenue and a 6,864% ROI over 17 months. On the eCommerce side, one audit surfaced 982 images with no size attributes causing layout shift, 590 images over 100KB slowing load, and 292 images missing alt text. Fixing the technical foundation there, including image and mobile work, helped lift the technical health score from 82 to 99 and grow monthly organic revenue from $77,000 to $147,000, for $968,000 in cumulative revenue over the campaign. The image counts are mundane. The revenue they were blocking was not.

Technical SEO and AI Search: The Critical Connection

AI Overviews, ChatGPT, Perplexity and Microsoft Copilot have created a dual-track reality. You now need to be technically optimised for both traditional Google indexing and AI retrieval systems. Traffic from generative AI platforms is growing quickly. Adobe reported that AI-driven referral traffic to US retail websites increased more than tenfold between July 2024 and February 2025, and Forrester has reported that 89% of B2B buyers now use generative AI in at least one phase of the buying process.

The technical requirements that help AI platforms cite your content overlap heavily with good traditional SEO, with a few additions worth calling out:

• Server-side rendering for any content you want AI platforms to access reliably.
• Clean, accessible sitemaps with timestamps to signal recency.
• Comprehensive structured data using FAQPage and HowTo schema.
• Core content delivered without a JavaScript dependency.
• AI crawlers explicitly allowed in robots.txt by name.

A genuinely useful way to think about AI visibility is as a leading indicator rather than a revenue line. Citations in ChatGPT, Perplexity and AI Overviews tell you whether your content is being surfaced and trusted, often before that shows up in clicks and conversions. It belongs on the dashboard next to rankings, as a measure of reach, not as a number you bank. A newer signal worth watching, though still emerging and not yet a settled Google practice, is the rise of conventions like an llms.txt file and the broader "generative engine optimisation" conversation. We would not rebuild a site around either today, but they are worth tracking as the AI platforms formalise how they prefer to access content.

Technical SEO and AI SEO are not separate disciplines. They share the same foundation: a site that is fast, crawlable, structured and secure performs better in both Google Search and AI-generated answers. Our AI SEO strategy guide shows how to optimise for both together.

Frequently Asked Questions

What Is the Difference Between Technical SEO and On-Page SEO?

Technical SEO focuses on your site's infrastructure: crawlability, page speed, architecture, structured data and security. On-page SEO focuses on the content within individual pages, including keyword usage, headings, meta titles and body copy. Both are needed. Technical SEO ensures search engines and AI systems can access and process your pages; on-page SEO ensures those pages are relevant and authoritative for the queries you care about.

How Often Should a Technical SEO Audit Be Conducted?

For most established business websites, run a full technical audit at least annually and monitor key signals monthly. Larger, faster-moving or more complex sites may justify quarterly audits. You should also audit after any significant migration, CMS change or major redesign, and whenever you notice an unexplained drop in organic traffic or indexing coverage.

Does Technical SEO Affect Visibility in ChatGPT and Other AI Platforms?

Yes, but the requirements differ by platform. For ChatGPT search, make sure important pages are crawlable in HTML and not blocking OAI-SearchBot. For Perplexity, allow PerplexityBot if you want visibility there. For Google AI Overviews, the key requirement is that the page is indexed and eligible to appear with a snippet in Google Search.

How Long Does It Take to See Results From Technical SEO Fixes?

Simple fixes like resolving crawl errors or submitting an updated sitemap can show results within days to weeks as search engines re-crawl. Larger improvements like Core Web Vitals work or a structured data overhaul usually show measurable results within one to three months. Migrations with technical changes can take three to six months to fully stabilise.

What Tools Are Used for Technical SEO?

The essential toolkit includes Google Search Console, Google PageSpeed Insights, a crawler such as Screaming Frog or a site audit tool, Google's Rich Results Test and Lighthouse. For AI visibility, use the Search Console Performance report to track AI Overviews and AI Mode within overall search traffic, alongside dedicated AI citation tracking.

What Is Crawl Budget, and Why Does It Matter?

Crawl budget is the number of pages Google is willing to crawl on your site in a given timeframe. For large sites with thousands of pages it becomes a real constraint. If Google spends its budget on low-value pages like thin content, filtered URLs or duplicates, your important pages may be crawled and indexed less often. Managing it means removing or blocking low-value URLs, fixing redirect chains and strengthening internal links to priority pages.

The Foundation Everything Else Depends On

Technical SEO is what makes every other SEO investment work harder, from content and links to AI visibility. As search expands across Google, ChatGPT and AI Overviews, the sites that win are the ones search engines and AI systems can crawl, index and understand without friction. Get the fundamentals right and you give every page a stronger chance across every search surface that matters.

If you want every technical fix tied to a measurable growth outcome rather than a crawler severity score, explore our technical SEO consulting.