Teckel AI · Guide

What is an agent-ready website? An auditable guide

Updated: September 20, 2026 · By Teckel AI

An agent-ready website reduces barriers around crawling, extraction, identity, actions, and measurement for automated systems without abandoning the human experience. That preparation does not guarantee that a platform will find, cite, or recommend the site. You will also see it called an AI website or an AI-readable website.

The confusion comes from the builders. Wix, Durable, or Hostinger will generate a site with AI in minutes, and that solves who constructs it. An agent-ready website addresses a different problem: how to verify what different automated systems receive and can extract. Automated traffic mixes search, security, training, and other bots; it is not equivalent to AI agents and does not describe traffic on a specific site. This guide defines the term, gives you five tests to audit your site today, explains what remediation can cost, and puts 60 days of our own server logs on the table: 1,753 requests from 18 distinct AI agents, including what they actually read, how long they take to find a new page, and the 64 requests that were signed as AI crawlers and were not.

Is a site built with AI the same as an agent-ready site?

No, and that misunderstanding is the one that costs companies the most. AI builders optimize production: templates, generated copy, a site live in an afternoon. The result looks good in your browser, which is a human running JavaScript on a screen. Automated crawlers and AI agents may receive and process the site differently, so the response HTML, rendering behavior, crawl controls, and server logs need to be tested for each surface.

An agent-ready website is designed for that visitor without abandoning the human one. The buying question has changed: it is no longer just "how does my site look?", it is now "can an AI agent extract my value proposition, my pricing, and my contact details without help?". If the answer is no, record it as a barrier to investigate. It does not prove universal invisibility or that another company will be cited.

How big is AI agent traffic? 60 days of our own server logs

There is no universal figure, and vendor reports mix search bots, security scanners, training crawlers and user-initiated actions. So instead of quoting you a global percentage of the web, here is ours with the whole universe on the table: every request to teckel-ai.com between July 22 and September 20, 2026, 60 consecutive days, classified by user agent. One domain, one sample, and further down we tell you exactly where the method breaks.

In those 60 days, 18 distinct AI agents made 1,753 requests:

Three readings hold with these numbers, and none of them needs an outside benchmark:

• AI agents requested more pages than classic search engines did. Googlebot, Bingbot, Applebot and DuckDuckBot together made 1,575 requests in the same window. The 18 AI agents made 1,753, about 11% more. That does not mean they send more visitors. It means they are already the majority of the machine reads on this site.
• 301 of those 1,753 requests are user actions. That is 17%, and commercially it is the category that matters: ChatGPT-User, Claude-User, Perplexity-User and Mistral only appear when a person, at that moment, asked an assistant to open your site. That is not training, that is somebody evaluating you.
• The list of agents grows on its own. Five of the 18 were not in the log in July and appeared for the first time in September: DuckAssistBot and Mistral on the 7th, Perplexity-User on the 9th, Cohere and YouBot on the 11th. Build for today's list and in two months you are five short.

For scale: in the same 60 days there were 17,636 human requests, roughly one AI agent request for every ten from a person. Agent traffic did not replace human traffic, it was added on top.

And four method warnings that apply to our numbers and to anybody else's:

• Automation is not the same as AI traffic. A global bot share does not describe the audience of a specific site.
• JavaScript support varies. Google can process JavaScript, while other crawlers and agents expose different capabilities. Serving essential content in HTML reduces a dependency, but a React application is not automatically invisible.
• Crawling, citation, and referral are different events. A server request does not prove that content was cited or that a person visited.
• Conversion belongs in your own funnel. A vendor benchmark cannot replace linked sessions, leads, opportunities, contracts, invoices, and settled payments.

A useful diagnosis combines logs, analytics, repeated observations, and commercial evidence. The question is not how many bots exist on the web, but what happens on your domain and what can be attributed without guessing.

The English side of a bilingual site: 15% of the URLs, 29% of the AI reads

This part of the measurement only appears when you split the same log by language tree, so it is not in the Spanish version of this guide. teckel-ai.com sells into Mexico, is written mostly in Spanish, and keeps its English pages under /en.

Excluding robots.txt, sitemap.xml and llms.txt, AI agents made 1,148 page requests in those 60 days: 813 to the Spanish tree across 148 URLs, and 335 to the English tree across 27 URLs. The English tree holds 15% of the addresses and takes 29% of the reads, which is 12.4 reads per English URL against 5.5 per Spanish URL, better than double.

The user action split says the same thing in commercial terms: of the 288 page requests that happened because a person asked an assistant to open something, 73 landed on English pages, which is 25%.

Read it with the caveat it deserves. Part of that concentration is arithmetic: fewer URLs absorb the same crawl budget, and the English tree is led by the home page and the service pages, not by the guides. It also does not prove that English content gets cited more, because crawling and citation are different events. What it does show, on this domain, is that the machines do not treat a second language tree as a courtesy translation. If you run one and you have been maintaining it as an afterthought, the afterthought is taking three out of every ten machine reads of your site.

What agents read most is not your articles: three files take 46%

When you break those 1,753 requests down by path, the result is uncomfortable for anyone investing only in content. The three most read routes are not articles:

Robots.txt, home and sitemap add up to 810 requests, 46% of all AI agent traffic on the site. The most read page by machines is not a page, it is a text file of rules. Part of that is structural and it has to be said: robots.txt is requested at the start of every crawl session and by design it will win any volume comparison. The practical consequence does not change. If those three files are wrong, the agent has already decided something about your site before reading a single line of your copy.

The number that goes against our own practice: llms.txt received 3 AI agent requests in 60 days, all three from GPTBot and all three on the same day, August 11. Robots.txt received 415, which is 138 times more. We keep llms.txt and we will go on keeping it, because it costs twenty minutes and it is not in the way, but with this data it does not survive being sold to you as a visibility lever. If somebody quotes you for AEO and the first thing they offer is an llms.txt, ask for their own measurement.

How long does an AI agent take to read a new English page? Seven measured

This is the question that breaks most agency reports, because measuring the day after you publish does not measure your content, it measures the crawl queue. These are the English pages we published between August 26 and September 18, 2026, with the date of the first AI agent read in the server log:

The median is 3 days and the range runs from 1 to 8. None of them was read on the day it was published. The two English pages from July are left out on purpose: our logging started on July 22, so their first read is not measurable and putting them in the table would fake a number.

And here is the finding that only shows up when you split by language. On this domain, the first reader of an English page was never ClaudeBot and never PerplexityBot: Amazonbot got there first three times, GoogleOther three, GPTBot once. On Spanish pages published in the same window, PerplexityBot arrived first twice and ClaudeBot once. It is seven pages against five, we are not going to dress it up as a law, and the honest version is this: the crawler that finds you first is usually not the one from the engine where you want to appear, and which one it is may depend on the language you published in.

The operational consequence is simple. If you publish today and measure tomorrow, you are measuring the crawl queue and you will conclude your content does not work. The number is also not transferable as is: this is a single domain with a current sitemap, static HTML and an automatic notification to engines on every publication. Without that, the reasonable expectation is longer, not shorter.

User agents lie: 64 credential requests signed as GPTBot, ClaudeBot and twelve others

Everything above is classified by user agent, which is what every agentic traffic report on the market does, and that method has an error floor almost nobody publishes. Here is ours, measured in the same 60 day window.

There were 64 requests to credential paths that do not exist on this site, signed by 14 of the 18 AI agent identities and coming from 49 distinct IP addresses. The paths requested include /aws/credentials, /.azure/credentials, /.gcloud/credentials, /terraform.tfstate, /terraform.tfvars, /serverless.yml, /config/credentials.yml.enc, /@fs/app/rootkey.csv and several directory escape attempts toward /proc/self/environ. The declared user agents include GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot, Google-Extended, Amazonbot, Meta-ExternalAgent, CCBot, Bytespider, Mistral and Perplexity-User.

No legitimate crawler asks for an AWS key file. What you are looking at is a vulnerability scanner wearing the name of an AI crawler to get past filters. It is 64 of 1,753 requests, 3.6%, so it does not move the conclusions above, but it does set the confidence floor of the method: a user agent is a string anyone can type, and an agentic traffic table without origin verification is a table of what visitors said they were.

This is exactly what separates layer 3 of the standard, actionable, from the other three. Discoverable, readable and reported can be open to everyone. Actionable cannot: if you are going to let an agent query inventory, quote a price or book a slot, the door does not open on a name, it opens on verified identity. The two largest operators publish their IP ranges for that purpose. OpenAI lists them at openai.com/gptbot.json, openai.com/searchbot.json and openai.com/chatgpt-user.json. Anthropic lists them at claude.com/crawling/bots.json, and its support documentation puts it without hedging: "If a crawler has a source IP address on this list, it indicates that the crawler is coming from Anthropic." All four sources verified against the origin on September 20, 2026.

What does peer-reviewed research say about what moves visibility?

There is one academic paper worth knowing before you hire anybody: GEO: Generative Engine Optimization, by Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan and Ameet Deshpande, presented at KDD 2024. The authors define it as "the first novel paradigm to aid content creators in improving their content visibility in generative engine responses", and their headline result, in their own words, is that "GEO can boost visibility by up to 40% in generative engine responses".

Two things the paper does support, and they line up with what this page does: the tactics that moved visibility most in their tests were quoting experts verbatim and backing claims with statistics from a named source. It was not adding more keywords. And the honest warning, which the paper itself makes clear: that 40% is measured on their own benchmark against a generative engine they built, not on your site and not inside commercial ChatGPT. It is a direction of work, not a promise of outcome. If somebody offers you that 40% as a guarantee, they are selling you a laboratory number as if it were a contract.

What makes a website agent-ready? The 4 layers

At Teckel AI we formalized this as the AI Website standard: four layers, each audited separately. A site is agent-ready when it meets all four.

These layers are design and audit criteria, not guaranteed ranking factors. Their effect must be evaluated through controlled changes while preserving the baseline, raw answers, citations, and server logs.

How do you know if your site is already agent-ready? Five tests for today

You do not need anyone technical for the five. Ten minutes and you are out of the doubt:

• 1. Inspect the response HTML. Use "View page source" or curl yourdomain.com. If essential content depends on rendering, record it as a potential barrier rather than proof of universal invisibility.
• 2. Check your robots.txt. Review the rules that apply to the agents you intend to measure. Blocking can prevent that agent from crawling, but allowing it does not guarantee a citation or recommendation.
• 3. Look for your sitemap. Confirm that sitemap.xml exists and is current. If llms.txt exists, treat it as optional documentation, not a demonstrated ranking factor.
• 4. Ask the AI about your company. Open ChatGPT or Perplexity and type "what does [your company] do?". Record whether it describes you accurately, confuses the entity, omits facts, or presents unsupported claims. The observation does not establish the cause by itself.
• 5. Review server logs. Browser analytics does not cover every automated request. Logs provide a complementary signal, but a user-agent alone does not prove identity, intent, or a recommendation.

If you find two or more barriers, prioritize a technical audit. A final diagnosis requires observing multiple agents and distinguishing eligibility, extraction, citation, and recommendation.

Do you have to rebuild from scratch?

Not always. First audit the response HTML, HTTP status, crawl controls, canonical, sitemap, structured data, and server logs. Serving essential content in HTML can reduce rendering dependence, but JavaScript capabilities vary across engines. The starting point is evidence, not a universal rebuild rule.

How much does an agent-ready website cost?

At Teckel AI, AI Websites start at $2,500 USD and include the four layers of the standard: a site that is discoverable, readable, actionable, and reported. These layers improve technical readiness but do not guarantee citations, traffic, or sales. If you first want to know where your current site stands, the initial diagnosis is free.

teckel-ai.com is the living demo

This site applies part of the framework, and the tables above are its layer 4 in public: every page is served as static HTML with structured data, and we log server requests associated with user agents declared by known bots. Those hits can be spoofed, as the 64 fake ones above show, and they do not prove a citation, a recommendation, a human visit or a commercial outcome. Useful reporting keeps those layers separate, which is why nothing on this page claims that 1,753 machine reads turned into a single sale.

Frequently asked questions

What is an agent-ready website?

An agent-ready website reduces barriers around crawling, extraction, identity, actions, and measurement for automated systems without abandoning the human experience. That preparation does not guarantee that a platform will find, cite, or recommend the site. It is also called an AI website or an AI-readable website.

Is a website built with Wix, Durable, or another AI builder agent-ready?

Not necessarily. Builders solve how a site is constructed. A site prepared for AI Discovery aims to make content eligible, extractable, and attributable across different engines. No technology, structured data, or file guarantees that an answer engine will cite it.

What are the four layers of an agent-ready website?

A practical audit covers explicit crawl controls and a current sitemap, extractable HTML and consistent identity, documented actions, and server-side measurement. llms.txt can serve as optional documentation, but it is not a demonstrated ranking factor.

How do I know if my website is invisible to AI?

There is no single test. Inspect the received HTML, robots.txt, HTTP status, canonical, sitemap, and server logs, then repeat controlled queries across engines. Google can process JavaScript, while other agents have different capabilities, so conclusions require observations rather than an absolute rule.

Do I have to rebuild my site from scratch to make it agent-ready?

Not always. First audit what each agent receives, which blocks exist, and what evidence is missing. Serving essential content in HTML can reduce rendering dependence, but the right change depends on the platform and does not guarantee citations or recommendations.

How much does an agent-ready website cost?

At Teckel AI, AI Websites start at $2,500 USD and include the four layers of the standard: discoverable, readable, actionable, and reported. The initial diagnosis of your current site is free.

How much of my website traffic will come from AI agents?

There is no universal figure, and a vendor benchmark does not describe your domain. On teckel-ai.com, between July 22 and September 20, 2026, 18 distinct AI agents made 1,753 requests against 17,636 human requests, roughly one agent request for every ten from a person. In the same window classic search crawlers made 1,575 requests, so AI agents already account for the majority of machine reads on this site. Your own server log is the only way to know your number.

Can I trust the user agent to identify an AI crawler?

No. A user agent is a string anyone can type. In 60 days of logs on this site, 64 requests to credential paths that do not exist here arrived signed as 14 of the 18 AI agent identities we track, which is 3.6% of all AI agent requests. Verify origin by IP range instead: OpenAI publishes its ranges at openai.com/gptbot.json, openai.com/searchbot.json and openai.com/chatgpt-user.json, and Anthropic at claude.com/crawling/bots.json.

How long does it take an AI agent to read a new page?

On this domain, seven English pages published between August 26 and September 18, 2026 were first read by an AI agent after a median of 3 days, with a range of 1 to 8 days. None was read on the day it was published. Measuring the day after you publish measures the crawl queue, not your content. The figure is not transferable: this site has a current sitemap, static HTML and an automatic notification to engines on every publication.

Go deeper: AI agency vs AI consultant, which one your company needs, what our AI Websites include and the best AI courses in the world.