A visit shows up on my dashboard with a small note about where it came from. LinkedIn. An email. Sometimes just one word: Google.
Think about what that word skips. Somebody typed a question into a box. Out of everything on the web, Google put my page on their screen, and they clicked it. My counter only sees that last click.
My last series, I Was Here, was about what happens after someone arrives. This one goes back a step. How did they find the page at all?
It's two parts. This one is how Google decides what to show. The next one is how an answer engine like ChatGPT decides what to quote.
The counter said "Google"
I've always liked searching my own name on Google, just to see what it gives me. A lot of the time, nothing interesting popped up, even though I already had websites out there.
So I started wondering. How does Google actually know I built something? And how does it end up suggesting it to someone who searches for something at random?
I'd added a sitemap and the other usual files to my site because the checklists said to. I couldn't have told you what any of them did for Google.
So here's the whole trip in plain words, taken from Google's own documentation: what SEO is, the three things Google does with a page, and the parts you actually control.
SEO is not a trick
SEO stands for search engine optimisation. The name makes it sound like gaming a machine. Google's own starter guide puts it more simply: "helping search engines understand your content."
Picture Google as a librarian. You walk up and ask a question. The librarian doesn't hand you the answer. They point you at a few books they think will help.
For your book to get pointed at, three things have to happen. Someone has to find it. Someone has to file it. And when a reader asks, the librarian has to think it fits the question.
SEO is making those three things easier. That's all it is.
Google doesn't search the web when you search
When you press enter, nothing goes out to the web. Google answers from its index, a giant database it built earlier. That's why the results come back so fast. The reading already happened.
So the work comes in three steps, and two of them are finished long before you ask.
- Crawl: a robot finds the page.
- Index: Google reads it and files it.
- Serve: when someone searches, Google picks pages from the index.
A page can drop out at every step, and each one has a different fix. Google is upfront that none of this is guaranteed, even if you do everything right.
Interactive: The three steps of Google Search
A robot has to find you first
Crawling is Google's program, called Googlebot, fetching pages. There's no master list of every page on the web, so it has to go looking.
Mostly it follows links. It starts from pages it already knows, and every link is a door to somewhere new. You can also hand it a sitemap, a file that lists the pages on your site.
So a page that nothing links to, and no sitemap mentions, is a book in a room with no door. It can be the best page you ever wrote. The robot will never walk in.
Interactive: A robot following links
When Googlebot does arrive, it opens the page much like your browser would. It runs the JavaScript in a recent version of Chrome, because plenty of modern sites build their content that way. Anything behind a login, it never sees.
There's also a note you can leave at the door. robots.txt is a small file that tells crawlers which addresses to leave alone. Two things about it catch people out.
- It's a request. Googlebot and other reputable crawlers follow it, but nothing forces them to. It's the same deal as the browser signals in part 4 of the last series. The honest ones listen.
- It doesn't keep a page out of Google. If other sites link to a blocked page, its address can still show up in results, just without a description. To really stay out, you use a
noindextag or a password.
Found isn't filed
Getting fetched doesn't mean getting kept.
Indexing is Google reading what it fetched: the text, the title, the images and the videos. It works out what the page is about and stores that in the index.
Then it deals with copies. The same post can live at several addresses. With www, without www, with a tracking tag stuck on the end. To you that's one page. To a robot it's three.
So Google groups pages that look nearly identical and picks one as the canonical, the main version that shows up in results. You can suggest which one you want, and Google often sorts it out by itself.
Interactive: One post, three addresses
One thing you may have heard is wrong. Duplicate content isn't a penalty, according to Google. It wastes crawling and it can confuse readers. Copying someone else's work is a different matter.
Not every page gets filed, though. Low quality can keep a page out. So can a noindex tag, or a design that's hard to read.
And this takes time. Google says changes can show up in hours or take months, and suggests waiting a few weeks before you judge whether something worked. A new page missing from Google today may mean nothing is wrong.
The ranking happens when you ask
Now a person types a question. Google looks through the index and returns what it thinks is most relevant and highest quality. This is serving, and it depends on hundreds of factors, including your location, language and device. The same words can give you and me different results.
Google publishes a guide that names its ranking systems. It doesn't publish the weights, so any list of exact factors is someone's guess. But the names alone change how you think about it.
- Meaning counts more than matching words. Systems like RankBrain and neural matching connect words to concepts, so a page can rank for a question without using its exact words. That's why repeating a keyword doesn't help. Done enough, it breaks Google's spam rules.
- Links work like votes. PageRank treats a link from one page to another as a recommendation, and a recommendation from a well-recommended page counts for more. It was the idea Google started with. Today Google calls it one signal among many.
- Originals beat copies. Other systems favour the original source over pages that only cite it.
- Fresh pages win when it matters. A search for today's news wants new pages. A search for how yeast works doesn't.
- Results get some variety. Google hides near-identical results, and generally shows no more than two from one site at the top.
Interactive: Links as votes
There's one more worth keeping in your back pocket. With passage ranking, Google can judge one section of a page on its own, so a long post can surface for a narrow question it answers in a single paragraph. It matters a lot in part 2.
Some popular advice doesn't hold up. Google's starter guide says these don't matter, or barely do:
- The keywords meta tag. Google Search doesn't use it.
- Keywords in your domain name or URL.
- A target word count.
- The number and order of your headings.
And you can't pay for any of it. Google says it doesn't take money to crawl a site more often or rank it higher. Ads are a separate thing.
What you actually control
You can't make Google pick you. You can make sure nothing stops it.
- Link to every page you care about from another page on your site.
- Publish a sitemap and tell Google where it is.
- Check that
robots.txtisn't blocking what you want found. Usenoindexfor what you want hidden. - Make sure the content is there for someone who isn't signed in.
- Give each page one address. Redirect the others, or mark the canonical.
- Write a title that says what the page is.
- Answer a real question, in the words a person would use to ask it.
- Say something original, so people have a reason to link to you.
- Wait a few weeks before you decide it didn't work.
For developers: how to check each stepshow
To check crawling, open /robots.txt and /sitemap.xml on your live site. In Google Search Console, the sitemaps report shows whether Google could read yours.
To check indexing, use the URL Inspection tool in Search Console. It says whether a page is indexed and which canonical Google chose, and it shows the page as Googlebot rendered it. Redirects and a <link rel="canonical"> tag are strong signals, and a sitemap entry is a weak one. Google can still pick a different canonical.
To hide a page, use noindex in a meta tag or an X-Robots-Tag header. Don't also block that page in robots.txt. If Googlebot can't fetch the page, it never sees the noindex.
To get pages noticed sooner, my site pings IndexNow when a post goes live, which tells Bing and a few other engines right away. As far as I can find, Google isn't part of it, so Google still gets the sitemap.
What to remember from part 1
- When you search, Google answers from an index it built earlier. It isn't reading the web at that moment.
- A page has to be crawled, then indexed, then chosen. It can drop out at any of the three.
robots.txtis a request about crawling.noindexis what keeps a page out of results.- Ranking runs on many signals, and Google doesn't publish the weights.
- You can't force a ranking. You can remove what's in the way, then be the right answer.
The librarian can only point at books that were found, filed and fit the question.
What if nobody clicks?
Every step in this post ends the same way: a list of links, and a person who picks one.
Now think about the last time you asked ChatGPT something. You didn't get a list. You got an answer, maybe with a small link beside one sentence. Google does it too now, with an AI summary sitting above the links.
That's a different character. The librarian points you at books. This is a friend who has read them, tells you what they say, and may or may not mention where they read it.
So who decides what the friend says? And whose page gets the credit?
That's part 2.