What an AEO pilot should test in its first three months
A three-month answer engine optimisation (AEO) pilot should test one thing: whether work on a small, defined part of your business changes how often AI assistants name, cite and correctly describe you, and whether enquiries follow. That takes a baseline recorded before anything changes, one set of questions tracked daily, and rules for judging the result agreed on day one.
In this guide
What should an AEO pilot prove?
A pilot should prove two things: that the work moves your results on a small scale, and that the method will repeat on a larger one. Being visible isn’t the test on its own.
Test four things, in this order, because each depends on the one before:
- Can AI read your pages? Crawlers need access, and the pages need to be indexed and say plainly what you do.
- Does it name you? How often you’re named on the questions you’ve agreed, and on which engines.
- Does it describe you correctly? What the answers say about your prices, services and locations, checked against the facts.
- Do enquiries follow? Whether people who found you through AI get in touch, by form, phone or email.
The UK Government takes the same layered view of its own visibility. Its guidance for government communicators tells them to audit AI answers regularly, pair automated reports with manual checks and use more than one tool. Its example from the Office for National Statistics tracked citations and accuracy over 14 weeks.
Visibility is only the second of the four. A pilot that raises it without the other three hasn’t shown that a wider programme would pay. If you can’t yet see where enquiries come from, our AI search attribution setup is built for that part.
How do you choose what to pilot?
Pick one product or service line with clear commercial value and a sales cycle short enough to show enquiries inside three months. Keep the scope to that line’s pages and the questions customers ask about it.
Then decide how many questions to track. Sistrix found it takes around 70 different questions to get a visibility score within 5 points, and around 200 to get within 3. With fewer, a pilot’s result can be lost in the noise, so if one service line can’t supply 70 real questions, widen the scope a little rather than track a handful. Our guide to choosing the questions to track shows how to write them.
A pilot can run on a small group of pages, but it shouldn’t stop at your own site. In Tilio’s study of 44,587 AI answers, 96.5% of citations went to sites the brand didn’t own. A pilot limited to your own pages tests half the job, so include the review sites, directories and publications AI reads on that subject.
What baseline should you record before it starts?
Record where you stand before anything changes: how often you’re named and cited on the agreed questions, what the answers say about you, and where your enquiries come from. A baseline taken after work begins can’t show a change.
Eight measures cover it, and each has a place it comes from:
| Measure | Where it comes from |
|---|---|
| How often you’re named, by engine | The questions you track, run daily on each engine |
| Citations of your own pages | The sources listed in those same answers |
| Who’s named instead | The other businesses in the answers where you’re missing |
| Whether what’s said is accurate | Answers that mention you, checked against the facts you’ve confirmed |
| Brand searches | Search Console’s Performance report, filtered to your brand name |
| AI impressions | Search Console’s Generative AI performance report |
| AI Assistant visits | GA4’s AI Assistant channel |
| Enquiries by stated source | The answers to “How did you first hear about us?” on your forms and calls |
Search Console’s Generative AI performance report shows impressions in AI Overviews and AI Mode, and GA4’s default channels now include AI Assistant, so both belong in the baseline from day one.
Run the baseline daily for at least two weeks, so one odd day doesn’t set the starting line. Our own daily score on 82 answers a day ranged from 3.7% to 11.0% over three weeks with no change to what we tracked, which is why a single day can’t be the baseline.
A review of the data you already hold is a fair first step. Search Console, GA4 and any tracking you’ve run before show where you stand before a provider adds anything, and past AI answers can’t be collected after the fact, so whatever exists is worth keeping.
Have four things ready for the first meeting:
- Access to Search Console, GA4 and the website
- The agreed questions, or a first draft of them
- One named owner on your side
- The “How did you first hear about us?” question live on your forms, set up as our guide on how to check if customers really found you through ChatGPT describes
What should each month test?
Month one records the baseline and removes anything stopping AI from reading your pages. Month two changes the pages and outside sources that matter most, and month three measures what moved.
| Month | What gets done | What it tests |
|---|---|---|
| Month one | Record the baseline daily, and fix anything stopping AI reading your pages, such as a blocked crawler or a page that isn’t indexed | Whether AI can find and read you |
| Month two | Change the pages and outside sources that matter most for the piloted questions, leaving a few questions untouched | Whether the work changes who gets named and cited |
| Month three | Measure the piloted questions against the baseline and against the untouched ones | Whether the change is real, and whether enquiries followed |
Month one starts with access because nothing else works without it. Google says a page has to be indexed and able to show a snippet to appear in its AI features, and “There are no additional technical requirements.” OpenAI says a site that blocks its search crawler, OAI-SearchBot, “will not be shown in ChatGPT search answers”.
Stage the changes, and leave a few questions untouched for comparison. If the questions you worked on move and the untouched ones don’t, the change is more likely yours. If both move together, something else, such as a model update, is the likelier cause.
Early signs can come fast. Our own AI search attribution page entered our sitemap on 13 November 2026, and ChatGPT first cited it the next day. One citation isn’t a result, but it shows the engines have found the work.
Gain, our monthly AEO programme, runs its first three months the same way. Month one is about knowing where you stand, month two about making the important changes, and month three about seeing what’s changing, with the next priorities set by what the evidence shows.
How do you avoid judging a pilot on one good week?
Compare weekly averages across the whole set of questions, never single days or single answers. On around 100 questions, a move of less than 4 or 5 points isn’t yet a result.
Our own score shows why. Between 21 October and 9 November 2026, our daily visibility score ran from 3.7% to 11.0% on 82 answers a day, with no change to the questions we tracked. A pilot judged on its best day in that stretch would have looked like a breakthrough, and one judged on its worst like a failure. Neither would have been true.
Weekly averages smooth that out. Sistrix found that with 20 questions, a weekly average carries about 1.8 points of uncertainty, against 4.8 for a single day. Report the pilot as weekly averages from the first week of the baseline, and judge it on the trend. Our guide on how to sanity check an AI visibility score goes further on how big a move has to be.
How do you decide whether a pilot has earned a wider rollout?
Settle it with three questions agreed on day one. Did the piloted questions move more than the untouched ones, did the change hold for at least four weeks, and did any enquiry name an AI assistant?
Three yeses make a strong case for going wider. One or two make a case for another quarter on the same scope, with the gap fixed first. A pilot with none still tells you something useful: what to change before you spend more.
Before a pilot becomes a longer programme, agree four things in writing:
- The scope. Which services, questions, engines and markets the next phase covers.
- The same questions carried forward. Keep the pilot’s questions in the set so the trend runs unbroken, and report any new ones beside them.
- How often results are reported. Weekly averages for the team, and a monthly report that links to the answers behind it.
- Who owns the data. The accounts, the tracked questions and the dated answers should stay yours.
Our guide to what good AI visibility reporting looks like covers what that monthly report should contain.
How do you switch providers without losing your history?
Keep the accounts in your own name and take the tracking record with you. Ask for the full list of tracked questions, the dated answers behind the scores and the rules used to count a mention.
Your analytics should already be yours. In Search Console, an owner has full control of a property, including adding and removing users, so make sure the owner is someone in your business rather than your provider. In GA4, a property keeps its historical data and settings when it’s moved between accounts, so one set up in a provider’s account can move to yours without losing anything.
Two tracking methods rarely give the same score, even on the same questions. Run the old and new tracking side by side for two to four weeks, so you can explain any jump in the numbers as a change of method rather than of results.
Use a short hand-over list:
- The tracked questions, exactly as worded
- The dated history: every answer and score, not only the monthly summary
- The counting rules: what counted as a mention, a citation or a recommendation
- Content still in production, and where each piece has got to
- The date access ends, for every account and tool
Our guide on how to choose an AEO agency has the questions to ask whoever takes over.
Tilio, a UK AEO agency, runs the first three months of Gain on the same lines as this guide. Month one sets the baseline and checks whether anything stops AI reading your pages, month two makes the important changes, and month three compares what moved with where you started. The next priorities follow the evidence.
Frequently asked questions
Can we trial an AEO agency on a small group of pages?
Yes. Choose the pages for one service and the questions customers ask about it. Include the outside sites AI reads on that subject, because in Tilio’s study of 44,587 AI answers, 96.5% of citations went to sites the brand didn’t own.
How should we choose a start date around our busiest sales period?
Record the baseline in a normal trading month, not at the peak. A seasonal rush or lull changes enquiries on its own and makes before and after hard to compare.
How do we launch an AEO pilot alongside a website migration?
Record the baseline before the migration and redirect every old address to its new one. Don’t judge the pilot until the engines have re-read the new pages, which shows when the new addresses start appearing as citations in the answers you track.
Can we switch AEO agencies without losing our tracking history?
Yes, if the accounts are in your name and you ask for the record. That means the tracked questions, the dated answers and the counting rules. Run the old and new tracking side by side for two to four weeks, because two methods rarely give the same score.
What should we do if our current AEO provider will not export historical data?
Ask in writing for the tracked questions and results, and check what your contract says about your data. Start fresh tracking straight away, because past AI answers can’t be collected after the fact.
Can a UK AEO consultant oversee the transition between two digital agencies?
Yes. An independent lead can hold the question list, the accounts and the hand-over plan while the agencies change, so nothing is lost between them. It works best when that person reports to you rather than to either agency.
What should we check before migrating to a different AI visibility platform?
Check that it tracks the same questions, engines and country, and counts a mention the same way. Run both for a few weeks so you can explain any jump in the numbers.
Written by Jack F, co-founder of Tilio, a UK AEO agency that tracks how businesses appear in AI answers and works to improve it. More about Jack F