How to Test Whether Important Content Is Available in Raw HTML

Checking whether your important content is in raw HTML is one of the simplest technical tests in SEO, and one of the most revealing. Raw HTML is what the server sends before any JavaScript runs. It is what most AI crawlers, link preview bots and many SEO tools read. The rendered DOM is what a browser builds after scripts execute, and it is what users see. When the two differ, crawlers may be missing your headings, product details, prices, FAQs or internal links. This guide shows how to compare them methodically and decide what needs fixing.

Raw HTML vs Rendered DOM vs Crawlable Content

It helps to define the three things you are comparing:

• Raw HTML (page source): the response body from the server. View it with “View page source” or a command such as curl https://example.com/page/.
• Rendered DOM: the live document after JavaScript has run, cookies have loaded and APIs have returned data. Inspect it in developer tools or save it using a headless browser.
• Crawlable primary content: the part of the page that defines what it is about, such as the main heading, body copy, key facts, structured data and links to related pages.

Your goal is simple: crawlable primary content should be present in raw HTML.

Step 1: Define What “Important Content” Means for Each Template

Different templates have different essentials. Write them down before testing so the check is objective.

Template	Content that must be in raw HTML
Service page	H1, introduction, service details, pricing approach, FAQs, contact path
Product page	Product name, price, availability, description, specifications, reviews summary
Blog post	Title, author, date, full article text, images with alt text
Location page	Business name, address, phone, opening hours, area served
Category page	Heading, intro copy, links to products or subpages

Include internal links and structured data in every list. Links help crawlers discover pages, and schema helps machines understand them. See our structured data optimization approach for what to include.

Step 2: Test Content in Raw HTML

For each template, choose three to five representative URLs. Then:

1. Fetch the raw HTML with curl, or view source in a browser.
2. Search for a distinctive sentence from each important content element.
3. Record whether it is present, partly present or missing.
4. Check that headings use proper heading tags and that links are real <a href> elements.
5. Confirm the canonical tag, meta robots and structured data are in the initial response.

If you prefer a visual check, disable JavaScript in your browser and reload the page. Anything that disappears is not in raw HTML.

Watch for Placeholder Text

Some pages show “Loading…”, skeleton blocks or empty containers in raw HTML. These are clear signs that content depends on client-side requests.

Step 3: Compare Against the Rendered DOM

Save the rendered DOM for the same URLs using a headless browser such as Chrome with Puppeteer or Playwright, or use a crawler with JavaScript rendering switched on. Compare:

• Word count of main content
• Number and text of headings
• Number of internal links
• Presence of structured data blocks
• Key facts such as prices or phone numbers

Many SEO crawlers can crawl twice, once with rendering off and once with it on, and report differences automatically. Large gaps highlight the templates that need work first.

Step 4: Check What Real Crawlers Receive

Your own browser test is not the whole story. Real crawlers may be served different responses due to CDN rules, geolocation, A/B testing tools or bot protection. Check:

• Search Console URL Inspection for Google’s rendered HTML and screenshot.
• Server logs to confirm AI crawlers received a 200 response and a normal response size. Very small responses suggest a shell or a block page.
• User-agent testing with curl, using crawler user agents, while remembering that IP-based rules may treat your test differently from the real crawler.

Problems here often connect with access issues covered in our guide to fixing AI crawler indexing.

Common Reasons Content Is Missing From Raw HTML

• Client-side rendered frameworks without server rendering
• Reviews, prices or stock loaded from third-party widgets
• Tabs and accordions that fetch their content only when clicked
• Infinite scroll without paginated URLs
• Personalisation scripts that replace default content
• Consent banners that block scripts that build the page
• Lazy-loaded text, not just lazy-loaded images

How to Fix the Gaps

Move primary content into the server response through server-side rendering, static generation or prerendering of the key elements. Keep interactive features client-side, but include their essential information in HTML. For third-party widgets, ask the vendor for a server-side integration or output a static summary, such as average rating and review count, in the page itself. Our custom web development and web design and development teams usually start with the highest-traffic templates.

Why This Matters for AI Search

AI systems quote, summarise and cite text they can extract. If your best explanation sits in a tab that loads on click, a competitor’s simpler page will be chosen instead. Raw HTML availability is therefore not just a technical tidy-up. It decides whether your content can be selected at all, which is why it is a core part of AI content optimization and of our approach to why AI is not picking your content.

Make It a Routine Check

Add raw HTML tests to your release checklist. Whenever a template, framework, CMS plugin or third-party script changes, re-run the comparison on representative URLs. A five-minute test can prevent months of lost visibility.

Frequently Asked Questions

Is It a Problem if Images Load Lazily?

Not usually, as long as image elements and alt text are present in HTML and native lazy loading is used. Text should not be lazy-loaded.

Do I Need Server-Side Rendering for Every Page?

Not necessarily. Static generation or partial server rendering of the primary content is often enough.

Will Google Still Index Client-Rendered Content?

Usually, but with delays and occasional failures. Other crawlers may not index it at all.

Testing raw HTML is quick and gives a clear list of fixes. For expert help, book a technical SEO audit, read why your content is not showing in AI search, learn how to build an AI-friendly website, review our AI indexing case study, or explore how to optimize for AI crawlers and LLMs.