Why ChatGPT Doesn't Recommend Your Insurance Agency
The short answer: ChatGPT, Claude, and Gemini recommend agencies they can find, read, and trust. Agencies get left out when their website blocks AI tools, says too little about what they do and where, or doesn’t match what the rest of the web says about them. Most of these are fixable, and two of them take about ten minutes.
Ask ChatGPT for a good independent insurance agent in your town. It will usually name a few agencies, with a sentence about each.
Here are a few well-reviewed independent agencies in Round Rock:
- Competitor Insurance: shops 20+ carriers; clients mention fast claims help.
- Another Agency LLC: strong on business and commercial auto policies.
- Third Agency Inc.: family-run, known for annual policy reviews.
Ranking well on Google doesn’t guarantee a spot. I’ve seen an agency rank #1 on Google Maps in its town and go unmentioned when AI assistants were asked the same question.
Here are the six causes I check, starting with the quickest.
How do AI assistants choose which agencies to recommend?
When an assistant searches the web for an answer, it can only recommend agencies it can find and read. None of the AI companies publish a ranking formula, the way Google publishes its local ranking factors.
What we do know is what they need from you:
- A website their tools are allowed to read
- Clear pages that say what you sell and where
- Details that match across your website, Google listing, and directories
- Reviews and mentions on other sites that back you up
The sections below cover each one.
Is your website blocking AI tools?
Check this first, because it’s the only cause that can shut you out completely. Every website has a settings file called robots.txt that tells automated tools what they may read. Some agency sites I’ve audited block AI tools in it.
To check yours, add /robots.txt to the end of your web address. Here’s what blocking looks like, and the fix:
User-agent: *
Allow: /
User-agent: OAI-SearchBot
Disallow: /
User-agent: Claude-SearchBot
Disallow: /User-agent: *
Allow: /
# Optional: opt out of training
User-agent: GPTBot
Disallow: /The tools to look for:
| Tool | Company | What it does | If you block it |
|---|---|---|---|
| OAI-SearchBot | OpenAI | Finds sites for ChatGPT’s search answers | Your site won’t be shown in ChatGPT search answers |
| GPTBot | OpenAI | Collects content to train OpenAI’s models | Your content isn’t used for training |
| Claude-SearchBot | Anthropic | Indexes sites for Claude’s search results | Your site may show up less in Claude’s results |
| Claude-User | Anthropic | Opens a page when a Claude user’s question needs it | Claude may be unable to read your page for that user |
| ClaudeBot | Anthropic | Collects content that may be used for training | Your future content is excluded from training |
| Google-Extended | Controls use of your content for Gemini | Doesn’t affect Google Search, per Google |
If you see Disallow: / under one of the search tools, or under User-agent: * (which means every tool), your site is blocking them.
The fix: allow the search tools. Blocking the training tools is your call, and it doesn’t stop you from being recommended. OpenAI says robots.txt changes take about 24 hours to reach ChatGPT.
Can AI tools actually read your website?
Being allowed in isn’t the same as being readable. Some agency sites show content only after scripts run in a browser. Some tools that read your site may see a mostly blank page instead.
Warning signs I find in agency audits:
- A large site where Google has indexed only one or two pages
- Important pages, like quote or coverage pages, blocked from search engines
- Text that lives inside images, which tools can’t read
A quick test: search site: followed by your domain on Google. If only a page or two comes up, AI tools likely struggle too.
Does your website clearly say what you do and where?
AI answers are built from plain statements, so your site needs to make them. “We’re here for all your insurance needs” gives an assistant nothing to repeat. “We’re an independent agency in Round Rock writing home, auto, and business insurance with 20+ carriers” does.
Make sure your site states:
- That you’re an independent agency, and roughly how many carriers you work with
- Each line you write, with its own page
- Each town you serve, with something specific to say about it
- Who’s on your team, and how long you’ve been in business
Behind the scenes, your site should also carry structured data (code that labels your name, address, and services for machines). I’ve seen agency sites whose structured data placed them in the wrong city.
Does the rest of the web agree with your website?
Assistants trust what they can confirm in more than one place. If your website, Google listing, and directory profiles show different names, phone numbers, or addresses, you’re harder to confirm.
This is the same cleanup that helps you on Google Maps:
- One exact version of your agency’s name, address, and main phone number
- No duplicate Google listings from old offices or former names
- Profiles on the directories clients use, with matching details
My post on why agencies don’t show up on Google Maps walks through each of these.
Do clients talk about you online?
Reviews give an assistant something to say about you. When ChatGPT describes an agency as “known for fast claims help,” that kind of detail often traces back to what clients wrote.
An agency with five reviews, none from this year, gives it little to work with. Ask every client for a review at the same moments, like a new policy or renewal, and reply to each one. Here’s how to get more Google reviews.
Mentions on other sites help too: your chamber of commerce, local news, sponsorships, and community “best of” lists.
Do you answer the questions clients actually ask?
Assistants quote pages that answer a question directly. When someone asks “Do I need flood insurance in Austin?”, the agency with a clear, local answer on its website has a chance to be cited.
Good topics come straight from your phones:
- What a policy covers, and doesn’t, in your state
- Local risks: hail, flooding, wildfire, or whatever your area faces
- What changes when someone buys a home, starts a business, or adds a teen driver
Answer the question in the first sentence, then explain. That’s the part assistants tend to quote. (This is how I approach content for agencies.)
How to check where you stand
Ask the assistants yourself, the way a client would. Use ChatGPT, Claude, and Gemini, and try questions like:
- “Who are the best independent insurance agents in [your town]?”
- “I need home insurance in [your town]. Which agency should I call?”
- “Recommend an insurance agency near [a neighborhood you serve].”
Ask each question a few times. Answers change from one run to the next, so one miss isn’t a verdict. Missing every time is a signal.
Where to start
If you only do three things this month:
- Open your robots.txt and make sure OAI-SearchBot and Claude-SearchBot aren’t blocked
- Add one clear sentence to your homepage: what you are, what you write, and where
- Fix any duplicate Google listings and start asking for reviews
The first two take about ten minutes. If you’d rather hand it off, here’s how I work with insurance agencies.
Frequently asked questions
Does blocking GPTBot keep my agency out of ChatGPT?
Not from ChatGPT's search answers. GPTBot collects content for training OpenAI's models. OAI-SearchBot is the one that decides whether your site can appear in ChatGPT search answers, and OpenAI says sites that block it won't be shown there.
Can I let AI recommend my agency without letting it train on my website?
Yes. OpenAI and Anthropic use separate crawlers for search and for training. You can allow OAI-SearchBot and Claude-SearchBot while blocking GPTBot and ClaudeBot.
Does blocking AI crawlers hurt my Google ranking?
Not the Google-Extended setting. Google says Google-Extended doesn't affect a site's inclusion in Google Search and isn't used as a ranking signal. Blocking Googlebot itself is a different matter and would hurt you.
How long until ChatGPT notices a change to my website?
OpenAI says it can take about 24 hours for its systems to pick up a robots.txt change. New content and reviews have no published timeline and usually take longer to show up in answers.
Why does ChatGPT give a different answer every time I ask?
AI answers vary from one run to the next. Ask the same question several times, in each assistant, before deciding whether you're being recommended.
Capra Growth is a local marketing practice forindependent insurance agenciesand other local businesses, based in Austin, Texas and working with clients across the US.