Last time I went looking for how AI picks which businesses to recommend and found that it does not read your website at all. Every source cited was third party. That leaves an obvious follow-up: what is your site actually for, and can anything read it in the first place.
Here is the answer, and I have a client’s numbers to put underneath it.
When people picture search, they picture themselves. Someone types something, a list comes back, they pick one. So when a business owner is told their site needs work, they usually assume the problem is the words on the page.
Almost always, it is not. By the time a human being is choosing between options, four other decisions have already been made about your site, and you were not in the room for any of them.
The client sells physical products online, retail and wholesale, on WordPress and WooCommerce. Custom theme, about thirty plugins, a couple hundred products, and a large library of supporting how-to pages.
The complaint was ordinary and unhelpful. The site was not performing in search and nobody could say why. Rankings were soft, the catalog was missing from Google’s shopping surfaces entirely, and a year of paid traffic had been running over the top of it.
Nothing looked broken. The site was well designed, loaded quickly, and every plugin was installed and configured correctly. That is the hardest starting condition there is, because a settings audit finds nothing, a content rewrite spends a quarter, and both leave the actual cause untouched.
So we did not audit settings. We tested the site the way a search engine experiences it, from the bottom of the stack up.
1. Can it be found?
Search engines send out software that goes around collecting pages. Think of it as a very fast, very literal person walking the neighborhood writing down every address.
It only has so much time for your street. If half the addresses it tries turn out to be a locked door, a forwarding note, or an empty lot, it uses up its visit on those and leaves before reaching the pages you actually care about. It does not know which ones were the important ones. Nobody told it.
The first move here is never an opinion. It is a population test: take the list of pages the site submits for indexing, request every single one as an anonymous logged-out visitor, and tally the responses.
$ curl -s "$SITE/product-sitemap.xml?cb=$n" \
| grep -o 'https://[^<]*' | grep '/product/' \
| while read u; do
curl -s -o /dev/null -w '%{http_code}\n' "$u?cb=$n"
done | sort | uniq -c
81 200
69 302
5 404The site was submitting 158 URLs for indexing and 74 of them were dead ends. Roughly half the file was a promise it could not keep.
Three things about that matter more than the number.
It was invisible from inside. In the admin panel all 158 products looked perfectly healthy, because an administrator holds keys the wholesale role gate hands out and a shopper does not. Anyone clicking through the dashboard would have found nothing, forever. This is the part I want owners to hear: everything looks fine when you are logged in and clicking around your own site.
Nothing was misconfigured. The 69 redirects were wholesale-only variants, correctly hidden from retail. The 5 not-found pages were configurable bundles and an add-on item, correctly flagged as hidden. Every plugin was doing its job precisely.
It named the general disease. One plugin knew a product was restricted. Another knew how to build a sitemap. Neither had any idea the other existed. Plugins publish, they do not reconcile, and the gaps between correctly configured tools is where these problems actually live.
One more query turned that symptom into a rule we could write code against. A single field decides visibility, and 72 published products carried a restricting value in it. That is a rule, not a list, which means the fix covers products added next year too.
The site also had no robots file on disk at all. It was being generated at request time by two plugins filtering each other’s output, producing two competing instruction blocks with the second one empty. A third-party question-and-answer widget was separately appending parameters to product URLs, manufacturing a duplicate of the entire catalog.
Result: 80 URLs, every one returning 200.
The sitemap got smaller. That is the goal, and it looks alarming on a chart if nobody warns you.
2. Can it be understood?
Now the page has been collected, and something has to work out what it says.
A person looks at your product page and instantly knows the price, that it is in stock, that it has good reviews, and roughly what it tastes like. Software does not know any of that unless the page states it in a form built to be read by software. Most sites carry a second, invisible copy of their key facts for exactly this purpose.
When it is missing, nothing looks broken. The page is fine. Your customers cannot tell. But every search engine and every AI assistant is looking at a page that never told them the price.
One command answers whether that copy exists. On this site, a product page published exactly two things: a Review record and an Organization record. No Product. No Offer. No price. No availability.
The commerce platform’s product data was not being published at all, on any product page or any category page. The one review record present was floating alone, attached to nothing.
That single output explained four separate symptoms the client had been treating as unrelated tickets.
Commerce platforms publish their machine-readable product data through template hooks. This theme rendered its own product markup rather than calling them, so none of it ever ran. Meanwhile the review service was publishing a complete, valid review record and the platform a complete, valid product record, and because neither knew about the other the ratings and the products lived in separate objects. No listing could ever earn stars.
The way I said this to the owner, without it landing as an accusation: the room was drywalled over the outlets. The wiring is correct and the electrician did the job. Nothing was installed wrong. But nothing can be plugged in until the covers are cut, and until then the room is exactly as dark as if there were no wiring at all.
Speed belongs to this question too, for an unglamorous reason. Software gives your page a certain amount of time to finish loading before it gives up and judges what it has. A page still assembling itself gets judged half-built. Your customers, meanwhile, just leave.
What eight days actually moved
Four files, all installed as must-use plugins so they load unconditionally and roll back by deleting one file. No theme edits, no content re-entry, no new subscriptions, and no client meetings required to ship any of it.
BeforeAfterProduct sitemap URLs that resolve84 of 15880 of 80Pages eligible for shopping surfaces066Pages with product data recognized28105Pages carrying review stars0295How-to pages with structured data0305Invalid items reportedn/a0
Those 305 how-to pages were generated by parsing the content already on them, not re-entered by hand. Worth noting because the handoff documentation said there were twenty. The sitemap said 305. That single correction changed the entire approach, since the standard recommendation would have meant re-entering every step by hand.
These are eligibility and comprehension measures taken from Search Console, not from a third-party tool.
Here is the part most case studies leave out. Over the same 28 day window the site did 144,153 impressions and 843 clicks, against 134,389 impressions and 843 clicks in the prior period. Clicks were exactly flat, on 7% more impressions.
That is the honest state of play eight days after shipping, and it is what you would expect. The structural work landed in the last week of that window. Nothing has had time to move. What follows is a baseline, not an after.
The bug that nearly shipped looking like a success
Every change went to staging first, and on this engagement that discipline paid for itself once, visibly.
$ wp eval 'warm_all_ratings();'
warmed 0 of 144 products
$ wp eval 'warm_all_ratings();' # identical command, minutes later
warmed 44 of 144 productsThe review service was rate limiting us, and the first version of our code was faithfully caching those empty responses for a full day. Shipped to production as written, it would have published zero ratings and looked like it worked.
The fix was a retry, a rule never to overwrite good data with worse, and a much shorter cache life for empty answers. It is a small bug. It is also exactly the kind that reaches production silently and gets diagnosed six weeks later as a mysterious ranking problem.
Four things the evidence disproved
An investigation is only as good as what it rules out, and half of what got ruled out here was our own.
Meta descriptions are missing site-wide. Inherited from an earlier assessment and repeated to the client before it was checked. Products, categories and the homepage all had good descriptions. Only how-to pages lacked them. Corrected in writing rather than quietly dropped.
Google is not reading the review service’s data. Our own hypothesis. The Rich Results Test showed all nine reviews being read correctly. The data was fine, it was simply attached to the wrong object. Retracted the same day.
Block the plugin assets directory in robots. A common recommendation and an actively harmful one. Search engines render pages, and blocking those assets would have stopped the site rendering properly. Withdrawn before it shipped.
There are twenty how-to pages to handle. Per the handoff documentation. There were 305.
Checking your own advice against the live server is cheaper than explaining it afterward, and clients trust the correction more than they distrust the error.
3. Will it get picked?
Only now does a person enter the story, and they are giving you about one second.
The most useful thing to understand here is that you are not choosing what your entry looks like. Google writes it, using material from your page. Your title is a strong suggestion. Your description is a weaker one. Both get rewritten regularly.
What you can change is more powerful anyway, and it is not the wording. It is the size and shape of your entry.
A plain result is one blue line and two grey ones. A result carrying a star rating, a price, an in-stock note and a small photo takes up two or three times the space and simply looks like a more serious option. Same page, same words. Bigger object, with proof attached.
That extra size is not written. It is unlocked by the invisible copy of your facts from question two, which is why those two questions are worth answering in that order.
Here is where the measurement gets interesting, and where I have to be careful about what I am claiming.
The top 24 non-brand queries by volume produced 61,718 impressions and 92 clicks. A 0.15% click rate, at average positions between four and ten. That is 43% of all site impressions returning 11% of the clicks.
QueryImpressionsClicksCTRPositionTop category term28,238180.06%6.9Same term, longer form9,269120.13%8.3Second category term3,54530.08%8.4Third category term3,26580.25%8.9A wholesale-intent term75700.00%2.6
The easy story is that these listings lack the visual weight to be picked, which is exactly what the structured data work just changed. That story would sell well and I am not going to tell it, because a 0.06% click rate at position 6.9 is roughly an order of magnitude below any published benchmark for that position. That gap is too large to pin on listing quality alone.
There are two candidate explanations and the data I have does not separate them.
Either the listings genuinely lack the weight to be chosen, or a large share of those impressions sit on surfaces where a blue link barely appears at all. That wholesale term at position 2.6 with zero clicks on 757 impressions leans toward the second. Position 2.6 with nothing to show for it is not a listing problem.
Which is where the work that just shipped becomes useful for something other than the client. It is a natural experiment. If merchant listings and review stars move that click rate over the next six weeks, it was listing weight. If it does not move, those impressions were never clickable and the right response is to stop counting them as opportunity at all.
I will publish that result either way. The second outcome is the more useful one to know and the less comfortable one to report.
Two smaller items came straight out of the same read. Category pages carry a marketing headline in the position where a shopper expects to see the thing they searched for, so the match is invisible above the fold. And 59 of the 305 how-to pages remain unindexed despite now carrying correct markup.
That second finding is the honest boundary of this kind of work. Once the machine can read everything, whatever is still not chosen is not a markup problem, and no amount of further engineering changes it. That one is editorial. It needs better photography and more depth on the page, not code.
4. Does anyone vouch for you?
The last question is the oldest one, and it is the one nobody can do for you from a keyboard.
Other sites linking to you. Press. Being mentioned by name in places you did not pay for. And the quiet one that matters most: whether people search for your business by name instead of searching for the category and hoping.
I expected to describe this one rather than prove it. Then I split the query data by branded and non-branded, and it turned out to be the clearest number in the whole engagement.
ImpressionsClicksCTRBranded queries1,08217916.5%Non-branded queries143,0716640.46%
Branded search is 0.75% of impressions and 21% of clicks. Over the trailing twelve months it holds: 1.5% of impressions, 24% of clicks. A quarter of everything this site earns comes from under two percent of what it is shown for.
The single comparison that makes it concrete: the brand name as a query returned 473 impressions and 124 clicks. The top category term returned 28,238 impressions and 18 clicks.
One brand query out-clicks twenty-eight thousand category impressions by seven to one.
That is what people mean when they say a brand is worth something, and it is the first time I have been able to hand an owner the arithmetic instead of the adjective. When people start typing your name, everything above this gets cheaper and easier. It is the only kind of demand you can create rather than compete for.
Two caveats, because the number is flattering and flattering numbers deserve more scrutiny than the other kind. Branded clicks over the last 28 days ran about 19% below the trailing-year rate, which for this category in August reads as seasonality rather than decline, and I would not present it as a trend either way without a year-over-year cut. And the branded figures come from a regex on the brand name, so they undercount misspellings and product-name searches. Directionally solid, not exact.
There is a market for buying this. It does not work, it is well understood by the people it is meant to fool, and it turns a slow problem into a fast one.
Why the order is not optional
Each question multiplies the ones before it, which means doing them out of order wastes real money.
Had this engagement started where these usually start, with content and rankings, the writing would have gone onto pages that were still unreadable and the money would have been spent twice.
Working bottom up also made each step measurable. Fixing access made the legibility problem visible. Fixing legibility turned the click-rate gap into a work queue with numbers attached instead of a matter of opinion. Every layer made the next one cheaper to diagnose.
Beautiful writing on a page nothing can reach does not get read. Press coverage pointing at a page that forwards somewhere else is coverage thrown away. A perfect headline on a product nobody was looking for still loses.
It is the same reason you do not install the countertops before the plumbing is connected. The countertops are not less important. They are just useless in the wrong order, and you will pay to do them twice.
The cheap, boring work goes first, not because it matters more, but because it decides how much everything after it is worth.
Words you might hear
If someone uses one of these, here is what they actually mean.
Indexing. Whether a page is in the collection at all. Not in it means it cannot appear, no matter how good it is.
Crawling. The collecting itself. The walk around the neighborhood.
Structured data. The invisible copy of your facts, written for software. What unlocks stars, prices and photos in your search entry.
Schema. The shared vocabulary that invisible copy is written in. Used interchangeably with structured data.
Sitemap. The list of pages you hand over and ask to have considered. Only useful if everything on it actually works.
Rich result. A search entry with the extra parts attached. The bigger one.
Redirect. A page that forwards somewhere else. Fine once. Expensive by the hundred.
Core Web Vitals. Three measurements of how fast and how steady your pages feel to real visitors.
None of this makes your site rank. It makes your site eligible, understandable and worth choosing. That is a smaller promise, and it is the one that has to be true before any bigger promise gets a chance.
This catalog did not need to be rewritten. It needed the covers cut off the outlets, in the right order, by someone willing to test their own advice against the live server before recommending it.
I do this for a living, so weigh that accordingly. But the four questions are true whether you hire anyone or not, and the first one is cheap enough that there is no excuse for still failing it.
Run it yourself. Pull your sitemap, open twenty of the URLs on it in a private window, and count how many go nowhere. If more than a couple do, you have found question one before anyone charged you to look for it. Reply and tell me what you got.