AI search and SEO: what developers should do now

If you run a site, you've probably noticed the search results page looks different. Many queries now show an AI-written overview above the traditional links, and chat assistants answer questions without sending anyone to a website at all. That raises an uncomfortable question for anyone who writes online: does SEO still matter, and what should change?

Short answer: the fundamentals still matter a lot, some tactics matter less, and a few new things are worth doing. I'll separate what's well established from what's guesswork, because this area is full of confident claims that nobody can verify. Search products change frequently, so treat details here as a snapshot and confirm against the official guidance from Google and others.

What's still true#

Search engines, including their AI features, rely on the same web. To appear in results or be cited in an AI answer, a page generally has to be discoverable, readable by a machine, and worth showing. That's the same list as before.

Crawlability. If a crawler can't fetch your page, nothing else matters. Pages need to return a proper 200 status, load without requiring a login, and not be blocked by robots.txt or a stray noindex. Server-rendered HTML is the safest bet. Content that only appears after heavy JavaScript runs is at higher risk of being missed or delayed.

Indexability. Each page should have one canonical URL, a sitemap that lists the pages you want found, and no accidental duplicates. Google's Search Console shows which pages are indexed and why others aren't, and it's free.

Speed and usability. Pages that load quickly and work on a phone are easier to crawl and better for readers. Core Web Vitals are a published set of measurements for this.

Clear titles and descriptions. The <title> and meta description still shape how a result looks and whether people click. Write them for humans.

Links. Other reputable sites linking to yours remains a strong signal of value, and good internal linking helps crawlers and readers move through your content.

Helpful, original content. Google has repeatedly said its systems aim to reward content that helps people, and that it doesn't matter whether a person or a tool wrote it, only whether it's useful and not created mainly to manipulate rankings. Their guidance on this is public and worth reading in full.

What's changing#

Some of the shift is real and visible. Some is speculation.

More answers, fewer clicks, for certain queries. When an overview fully answers a simple question, fewer people click through. Publishers have reported traffic changes, and the size varies by topic. Simple factual queries are more exposed. Queries that need depth, opinion, tools, or a purchase are less so.

Citations as a new kind of visibility. AI answers often show links to sources. Being one of those sources can send visitors, and it builds brand recognition even when no one clicks. Nobody outside the companies knows precisely how sources are chosen, so be skeptical of anyone selling a formula.

Longer, more conversational queries. People ask assistants full questions and follow-ups. Content that answers a specific question directly, then goes deeper, fits that pattern well.

Quality bars rising for generic content. When anyone can generate a passable article in seconds, passable articles stop being valuable. The web is filling with near-identical AI-written pages, and search systems are actively working to demote thin, mass-produced content. Google's spam policies now explicitly cover scaled content abuse.

New crawlers. Companies that train and run AI models operate their own crawlers, and some assistants fetch pages live on a user's behalf. You now have to decide who you allow.

Be honest about "GEO"#

You'll see the terms generative engine optimization, AI search optimization, and a few others, often attached to a paid service. Some advice under those labels is sound: it's mostly good SEO and clear writing. Some is unproven folklore, like special phrasing that supposedly makes a model cite you. Since the underlying systems are opaque and change without notice, anyone claiming a guaranteed trick is guessing. A useful rule is to be suspicious of any tactic that would make your page worse for a human reader.

A practical checklist#

These are things you can do this week on a technical site, ordered roughly by payoff.

1. Make sure the basics work#

Check that key pages return 200, that the canonical tag points to the right URL, and that robots.txt isn't blocking anything by mistake. Submit a sitemap in Search Console and look at the indexing report. Fix pages marked "crawled, currently not indexed," since that often signals thin or duplicate content.

2. Put the answer first#

Structure articles so the direct answer appears near the top, then expand. A clear definition or a short summary under the heading helps human skimmers, search snippets, and AI systems that extract passages. Use descriptive headings that match how people phrase questions.

3. Add real expertise#

This is your best defense against generic content. Include things a tool can't invent for you: your own measurements, screenshots, code you actually ran, a failed experiment, a decision and its reasoning, and named sources. Original data and first-hand detail are the most defensible asset you have. If you haven't done something, don't write as though you have. Made-up experience is worse than none, and readers can tell.

4. Use structured data#

Schema.org markup in JSON-LD helps machines understand a page. For articles that means Article or BlogPosting with author, dates, and headline. Add BreadcrumbList for navigation, and Person or Organization for identity. Validate with Google's Rich Results Test. Structured data doesn't guarantee a fancy result, but it removes ambiguity.

5. Show who's behind the content#

Add a real author name, a short bio, an About page, and contact details. Keep publish and update dates accurate. Trust signals matter more for topics where mistakes could hurt people, such as health, money, or security.

6. Decide on AI crawlers#

robots.txt lets you allow or block specific crawlers by name. Some companies publish separate user agents for training data collection and for live browsing or search, and the policies differ. Check each vendor's documentation for the current names. A sample that blocks one training crawler while allowing everything else looks like this:

User-agent: GPTBot
Disallow: /

User-agent: *
Allow: /

There's no universally right answer. Blocking training crawlers protects your content from being used to train models, and may or may not affect your visibility in AI answers, depending on the vendor. Blocking search-related crawlers can remove you from those results. Decide based on your goals, and revisit it as policies change. Note also that robots.txt is a request, and only well-behaved crawlers honor it.

7. Keep pages fast and simple#

Compress images, serve modern formats, enable caching and compression on the server, and avoid layout shifts. Measure with PageSpeed Insights or Lighthouse. Speed helps people first, and machines second.

8. Build internal links and topical depth#

Write a cluster of related articles, and link them to each other with descriptive anchor text. A site with twenty connected, thorough pieces on one subject looks more authoritative than one with a hundred unrelated posts. Update old posts as things change, and say so on the page.

9. Earn real mentions#

Share your work where relevant communities gather, answer questions in forums with links only where they help, contribute to open source, and speak or write for others. Links and mentions from real people are hard to fake, which is exactly why they carry weight.

10. Measure, then adjust#

Look at Search Console for impressions, clicks, and queries. Track which pages get traffic and which sit idle. If your analytics allow it, look at referrals from AI assistants, which some now show as distinct sources. Don't chase daily fluctuations. Compare month over month.

Thinking about queries in a new way#

Keyword research used to mean finding the phrase people type and matching it. That still works, but it helps to think in questions and tasks. What is the person trying to get done, and what would they ask next? A reader searching "what is prompt caching" probably wants a definition, then an example, then the catch. A page that covers that sequence in order serves them better than one stuffed with the phrase.

A good exercise is to take one article and write down the five follow-up questions a reader would have after finishing it. If your page answers none of them, it's a summary of what everyone else says. If it answers them, or links to pages of yours that do, it's a stronger resource for both people and the systems that pick passages to quote.

Pay attention to your own search data too. Search Console lists the queries that already bring impressions to your pages. Queries where you show up on page two are often the cheapest wins, because a better title, a clearer opening, or one added section can push them onto page one. Improving an existing page usually beats writing a new one.

A note on using AI to write your content#

Using AI as a writing aid isn't banned by search engines, and plenty of good publications use it for research, outlines, and editing. The danger is scale without substance: dozens of near-identical articles with no original input. Google's public policy targets that pattern regardless of how the content was produced.

If you use a model, do it the way you'd use a junior assistant. Give it your notes, data, and opinions. Check every fact and every code sample yourself. Rewrite passages that sound generic. Add what only you know. And publish fewer, better pieces instead of more, faster ones. That's also the best way to stay eligible for ad programs, which have their own quality reviews for sites with thin content.

For developers, a few technical extras#

Return correct status codes, since a soft 404 that says "not found" but returns 200 confuses crawlers. Use rel="canonical" on paginated and filtered pages carefully, and give paginated lists clean URLs. Add an RSS or Atom feed, which some readers and crawlers still use to find new posts. Keep the sitemap's lastmod values honest. Make sure your 404 page has noindex, and your admin pages aren't crawlable. Avoid hiding main content behind tabs that only load on click.

If you serve a lot of documentation, consider offering a plain-text or Markdown version of key pages. Some teams also publish an llms.txt file describing their site for AI tools. It's a proposal, not a standard, and it's unclear how widely it's used, so it's optional and low effort at best.

What not to do#

Don't stuff keywords, since modern systems recognize it and readers hate it. Don't publish hundreds of auto-generated pages to capture long-tail queries. Don't buy links. Don't cloak, meaning show crawlers something different from what users see. Don't fake author profiles or invent credentials. Each of these can earn a manual action or a slow demotion, and recovering takes far longer than the shortcut saved.

The durable takeaway#

Underneath the new vocabulary, the advice hasn't moved much: make a site that's easy for machines to read and genuinely useful to people, and show that a real person with real knowledge stands behind it. What has changed is the penalty for the generic middle. When a model can produce an average answer for free, the pages that still earn visits are the ones that offer something an average answer can't: a tested result, a clear opinion, a working example, a perspective from someone who did the work.

Do the technical basics once, keep them clean, and then spend your effort on the part nobody can automate away. That's a more reliable plan than chasing each change in the results page.