Make JavaScript pages work as documents

Two readers, two behaviors

A browser shows you the page after JavaScript has run. A crawler may see something else. Google describes a two-step process: Googlebot fetches the URL, reads the HTML and its links, then queues pages that return a 200 status for rendering, where a headless Chromium runs the JavaScript. Google then indexes the rendered HTML and queues any new links it finds.^[1] Rendering can wait in the queue, and Google notes that server-side or pre-rendering remains a great idea because it makes a site faster for users and crawlers, and not all bots can run JavaScript.^[1]

Google’s documentation on dynamic rendering says the same about other engines: they may choose to ignore JavaScript and then cannot see content that JavaScript generates.^[2] The AI systems are in that group more often than not. A December 2024 analysis of crawler traffic on the Vercel network found that OpenAI’s crawlers, Anthropic’s ClaudeBot and PerplexityBot did not execute JavaScript, while Googlebot and Apple’s crawler did.^[3] Those are observations from one platform, taken at one time, and the vendors do not publish promises either way. The prudent reading is to avoid depending on it.

What Google does with your scripts

Most JavaScript problems are not about the framework. They come from a handful of habits, and Google’s guide names them. Links must be <a> elements with an href, and single-page apps should use the History API, not URL fragments, to move between views.^[1] You may set the title, meta description and canonical with JavaScript, but the canonical should not differ from the one in the original HTML, and HTML is the better place for it.^[1] Client-side routing makes meaningful status codes hard, so Google suggests either redirecting to a URL the server answers with a 404 or adding a noindex robots meta tag to error pages, to avoid soft 404s.^[1]

Google’s guidance for mobile-first indexing adds a related rule. Google indexes the mobile version of a page, so the mobile page needs the same primary content, metadata and structured data as the desktop page, and should not hide primary content behind interaction.^[4]

Inspect the public route directly

You can find most of these problems in five minutes without any tool beyond a terminal and a browser. The goal is to compare three versions of the page: what the server sends, what the browser builds and what Google reports.

# 1. What the server sends, before any JavaScript
curl -s https://example.com/services/sports-massage/ | grep -i -E "<h1|canonical|robots"

# 2. The response code for a URL that should not exist
curl -s -o /dev/null -w "%{http_code}\n" https://example.com/no-such-page-xyz

If the first command prints nothing, the headline and metadata are created in the browser. In Chrome you can also disable JavaScript in DevTools and reload to see what remains. Then ask Google. The URL Inspection tool and the Rich Results Test render the page and show the loaded resources, the JavaScript console output and exceptions, and the rendered DOM, so you can check that the rendered HTML has all the content you expect.^[6]

Choose a rendering strategy on purpose

The framework matters less than how the first response is produced. Google recommends server-side rendering, static rendering or hydration over dynamic rendering.^[2]

For a typical studio or service-business site, static generation or server rendering is the simplest answer. The content exists as HTML, and JavaScript adds animation, filters and forms. This is also what makes pages fast, which the page experience chapter covers.

Test after every release

Rendering problems usually appear after a deploy, when a new component, a lazy-load change or a router update silently moves content out of the initial HTML. A tiny script that runs after each release catches this early. This one checks the response code, the headline, the canonical and the robots tag, and confirms that an unknown URL returns 404.

// check-pages.mjs  run with: node check-pages.mjs
const base = 'https://example.com';
const pages = ['/', '/services/sports-massage/', '/guides/hamstring-strain/'];
for (const path of pages) {
  const res = await fetch(base + path);
  const html = await res.text();
  const text = html.replace(/<script[\s\S]*?<\/script>|<[^>]+>/g, ' ').replace(/\s+/g, ' ');
  const checks = {
    'status 200': res.status === 200,
    'has h1': /<h1[\s>]/i.test(html),
    'has canonical': /rel=["']canonical["']/i.test(html),
    'not noindex': !/name=["']robots["'][^>]*noindex/i.test(html),
    'has text in the HTML': text.length > 300,
  };
  for (const [name, ok] of Object.entries(checks)) console.log(ok ? 'ok  ' : 'FAIL', path, name);
}
const missing = await fetch(base + '/no-such-page-xyz');
console.log(missing.status === 404 ? 'ok  ' : 'FAIL', 'unknown URL status', missing.status);

It does not replace a real audit. It does turn “the page disappeared from the HTML” from a surprise in Search Console three weeks later into a failing check on release day. Add the pages that matter most, and extend the script when a new failure teaches you something.

Apply it to your website

• The initial HTML of every important page contains the h1, the main text, internal links, the title, the canonical and any robots meta.
• Navigation uses real anchor elements with real URLs, and routing uses the History API and not fragments.
• Unknown URLs return a real 404, or the app redirects to one, or it adds noindex to error views.
• Nothing important loads only after a click, a tab change or a scroll event.
• The mobile page has the same primary content, metadata and structured data as the desktop page.
• Public content uses server rendering, static rendering or hydration, and no bot-specific version is served.
• A post-deploy check confirms the initial HTML of key pages and the response code for an unknown URL.