AI answers vary from run to run, so one screenshot proves little. Use a fixed question set, record the service, date and settings, repeat each question several times and report how often you appear, with the number of runs beside every percentage. Count mentions, citations and referrals separately, and use the findings to fix your pages, not to chase a rank.
Define the question set
An audit is only as good as its questions. Use the questions customers really ask, taken from your keyword work, enquiries and sales calls, written the way a person would type or say them. Fix the wording and keep it. Changing the questions every month makes any trend meaningless. A set of 20 to 40 is enough for a small business; more adds effort faster than it adds insight.
- Category
“How long does a website redesign take?”
Do you show up when nobody names you? - Comparison
“Site builder or custom website for a clinic?”
Are you present when people choose? - Local
“Web studio in Leeds for a small business”
Does the answer know where you are? - Brand
“What is Harbor Studio, and who runs it?”
Is what it says about you accurate?
Tag each question with its intent, its group and the page that should answer it. Keep a short list of control questions about topics where you should not appear. If you show up for those, something is wrong with the method.
Preserve the observation context
Answers depend on the service, the mode, the location, the account and the moment, so each observation needs its context. Record what you can and note what you cannot control. Without these fields you will not be able to tell whether a change came from your pages or from the setup.
| Field | Example (illustrative) | Why it matters |
|---|---|---|
| Question ID and exact text | Q07: “How long does a website redesign take?” | Ties the run to the fixed set. |
| Service, mode and version | ChatGPT, search on, model name shown | The same question differs by service and mode. |
| Date, time, location, language | 2026-10-19 10:12, UK, English | Results change over time and by place. |
| Signed in or out | Signed out, fresh session | Personalization can shift answers. |
| Run number | 3 of 5 | Lets you measure variation. |
| Full answer and cited URLs | Text saved, 4 links listed | Evidence for scoring later. |
| Result | Cited, mentioned only, absent | The thing you will count. |
Save the answer text and a screenshot, not just your impression of it. A browser profile used only for audits, with the same settings each time, removes one source of noise.
Run each question more than once
The strongest reason for a method is the variation. SparkToro and Gumshoe.ai, as reported by Search Engine Land, had 600 volunteers run 12 prompts through ChatGPT, Claude and Google’s AI features a combined 2,961 times in late 2025. They found that repeated prompts rarely gave the same list of brands, with less than a one-in-100 chance of an identical list from ChatGPT or Google’s AI, and that the order and number of recommendations also shifted.[1] They concluded that the percentage of responses in which a brand appears was steadier than its position in the list.[1] The study was run with a company that sells AI tracking tools, was not peer-reviewed and used volunteers, so treat it as a strong warning and not as a precise measurement.[1]
The practical result is to run each question several times, five or ten if you can, and to report how often you appear. Report the denominator every time. Statistics for small samples are blunt: if you appear in 4 of 20 runs, that is 20%, but a standard 95% confidence interval for that observation runs from roughly 8% to 42%. A move from 20% to 30% over a month in a sample that size is not a trend.
| Question group | Runs | Cited | Mentioned only | A named competitor cited |
|---|---|---|---|---|
| Category | 50 | 6 (12%) | 4 (8%) | 19 (38%) |
| Comparison | 30 | 3 (10%) | 2 (7%) | 11 (37%) |
| Local | 30 | 9 (30%) | 5 (17%) | 8 (27%) |
| Brand | 20 | 17 (85%) | 3 (15%) | Not applicable |
The numbers are invented. The format is the point: counts and percentages together, grouped by question type, with competitors alongside.
Distinguish the events
“Visibility” covers several different events, and they are measured in different places. Do not add them together.
- Mention
Your name appears, with no link.
Seen only in your own audit - Citation
Your page is shown as a source.
Audit, Bing AI Performance - Impression
A link to you was shown to someone.
Search Console AI report - Visit
Someone clicked through.
Analytics referrals
Google’s generative AI performance report in Search Console counts impressions: how many times links to your site were shown to a user in AI Overviews and AI Mode, by page, country, device and date. Its documentation lists impressions only, and the data also remains inside the regular Performance report, so adding them together would double-count.[2][3] Bing Webmaster Tools’ AI Performance, in public preview from February 2026, reports how often your content is cited in Microsoft Copilot and Bing’s AI summaries, which URLs are cited and how that changes over time. It reports citations and not visits.[4] For visits, OpenAI says ChatGPT adds utm_source=chatgpt.com to referral URLs, which lets you identify those visits in analytics.[5] The next chapter brings these into one review.
Turn findings into source improvements
The audit is worth running only if it changes pages. For each question where you expected to appear and did not, work through the likely causes, from the cheapest to check to the hardest.
| Finding | Likely cause | Action |
|---|---|---|
| Absent for a question your page answers | Page not indexed, blocked, or unreadable without JavaScript | Check crawl, rendering and crawler controls first. |
| Competitors cited, and you are not | Their page has evidence, structure or specificity yours lacks | Compare the pages. Add what is missing and true. |
| Cited, but the answer misstates you | An ambiguous or outdated passage on your page, or a conflicting directory entry | Fix the source, then re-run later. |
| Wrong company or facts | A similarly named business or an inconsistent identity | Strengthen identity signals. See the brand chapter. |
| Appears for control questions | A flaw in the question set or method | Revise the method before drawing conclusions. |
Make one change at a time where you can, date it and re-run the same questions after a few weeks. Remember that fixing a page does not force another system to update, and that a single good run proves nothing. Be skeptical of tools that promise to explain why an AI service chose a source. Google states that no third-party tool has access to its internal ranking or AI systems.[6] A tool can sample answers and count, which is useful, but it cannot see inside them.
Keep the practice small and honest
A spreadsheet and a monthly session are enough to start. Run the full set quarterly if monthly is too much. If you buy a tool, ask how many times it runs each prompt, which services, modes and model versions it covers, from what location and whether it is logged in, how it stores answers and how it handles variation. A vendor who cannot answer cannot support a claim about your “AI share of voice.”
Audit protocol
- Question set
- 20 to 40 fixed questions, tagged by group, intent and answering page.
- Services and modes
- The exact services and settings, kept the same each time.
- Runs
- At least five per question, with the date range.
- Record
- The log fields above, plus the saved answer.
- Report
- Counts and percentages by group, with competitors and the confidence caveat.
- Actions
- Source fixes, owners and a re-run date.
Apply it to your website
- You keep a fixed set of 20 to 40 questions, grouped by type and tagged with the page that should answer each.
- Every observation records the service, mode, date, location, sign-in state and the full answer with cited URLs.
- Each question is run several times, and results are reported as counts with their denominators.
- Mentions, citations, impressions and visits are counted and reported separately.
- Changes smaller than the run-to-run variation are not reported as trends.
- Gaps are traced to causes and turned into dated changes on your own pages.
- Any tool you use can explain how many runs it makes, with which services and settings.
Sources and further reading
Sources were consulted in October 2026 and the numbered notes above point here. Platform documentation changes often, so check the linked page before you act on a platform-specific detail. Worked scenarios are illustrative unless identified otherwise; methods and worksheets are Vantle’s editorial guidance.
- Search Engine Land: AI recommendation lists rarely repeat (SparkToro and Gumshoe.ai study)Coverage of the study of 2,961 repeated prompts; the vendor relationship and limits are described there.
- Search Console Help: Generative AI performance report (Search)What the report counts, its dimensions and how impressions relate to the Performance report.
- Google Search Central Blog: Introducing Search Generative AI performance reports in Search Console (3 June 2026)The announcement of impressions, pages, countries, devices and dates.
- Bing Webmaster Blog: Introducing AI Performance in Bing Webmaster Tools, Public Preview (February 2026)Microsoft’s announcement of citation reporting for Copilot and Bing AI summaries.
- OpenAI Help: Publishers and developers FAQReferral tracking with the utm_source=chatgpt.com parameter.
- Google Search Central: Optimizing your website for generative AI featuresThe note that no third-party tool has access to Google’s internal ranking or AI systems.