Rise of Machines

Come with me if you want to convert. Judgment Day came and went, and it turns out the machines just want to know the price of your software.

The machine uprising is real. We just replaced the T-1000s with AI agents. And machines are already most of the traffic on the web, which means the site you built for people is now being read by something that isn’t one. Agents don’t skim your copy, they run your site. This isn’t an uprising to resist, though. The companies that take advantage early are the ones that will benefit most from this new era.

Here’s the number that should get your attention. HUMAN Security’s 2026 State of AI Traffic report clocked traffic from AI agents and agentic browsers growing 7,851% year over year across 2025. Not bot traffic generally, agents specifically. Broader AI-driven traffic grew 187% from January to December, while human traffic grew about 3%. And in June, Cloudflare’s Matthew Prince put a marker down: automated requests crossed 57.5% of HTML traffic against 42.5% human, the first machine majority on record.

The opportunity isn’t that nobody’s working on AI agents. It’s that most of them are not doing the right things. GNW Consulting and Demand Metric surveyed 225 B2B marketing leaders and found 92% already experimenting with or operationalizing AI-search work. But fewer than 15% have put one person in charge of that work, and a lot of what has shipped so far went into files almost nothing reads.

Two kinds of machine visitor

An AI agent isn’t a person reading your site in a browser tab. It’s software, and it gets onto your site one of two ways.

The first is a crawler. A crawler requests the page, reads the raw HTML that comes back, and stops there. Vercel and MERJ studied hundreds of millions of fetches and found GPTBot, ClaudeBot and PerplexityBot download JavaScript files without ever executing those files. A pricing table assembled in the browser after load therefore reaches a crawler as an empty div. Googlebot and Applebot render JavaScript. The AI crawlers, by and large, do not.

What a person sees

Pro — $49 / user / month
SAML single sign-on · Audit log export · Salesforce sync
Start free trial

What the crawler gets
<div id=”pricing-root”></div>
<script src=”/app.bundle.js”></script>

No price. No tier name. No integrations. Nothing to quote you on.

Same page, same moment. Content assembled client-side arrives as a blank to any crawler that never runs the script.

The second is an agentic browser, which is closer to a robot driving a real browser window, clicking buttons and filling out forms the way a person would, except there’s nobody there. Comet, Atlas, Claude in Chrome. These do run your JavaScript. What they struggle with instead is the interface itself.

What agents actually take from your page

Either way, it didn’t show up to browse. Somebody asked it to do a job, check a price, see if you support a specific integration, pull a spec sheet, and it’s trying to finish that one thing and move on. It is not reading your hero copy. It is not seeing the banner or the brand story you spent months getting right.

That’s the part that trips people up, so let me be precise about it. Agents skip your persuasion. They do not skip your facts. The narrative arc, the emotional build, the reason someone should feel good about you, all of that gets stepped over. But the pricing tier, the integration name, the customer result, the compliance certification, those get pulled out and carried straight into somebody’s comparison doc. Content still matters, don’t get me wrong. Plenty of people will keep landing on the page and reading it. But a fast-growing share of that traffic is agents scraping the facts of your business, and clean, machine-legible facts now deserve as much of your time as the marketing message does.

One product page, as an agent reads it

[skipped]  The category we helped invent
[skipped]  For fifteen years we’ve believed procurement should feel human again.
[quoted]   Pro tier, $49 per user per month
[quoted]   SAML single sign-on, SCIM provisioning, audit log export
[skipped]  Our customers call it the most delightful software they’ve ever rolled out.
[quoted]   Salesforce, HubSpot and Slack integrations, generally available
[quoted]   SOC 2 Type II, renewed March 2026
[skipped]  Book a demo and let’s build something together.
[quoted] = quoted into a comparison doc · [skipped] = never read

The marked lines are the ones that survive the trip into somebody’s vendor shortlist. Every line above took the same effort to write.

Where most sites fall down

A button that just says “click here,” with nothing in the code saying what the button does, a person can figure out on sight. A machine cannot, because there’s no accessible label behind it. Researchers who tested this on a cloned news site found agents never scrolled past two viewports, and a pixel-only “Order Now” button drew zero clicks out of ten. Swap in a real HTML button and the clicks started.

Writing has the same problem. If I say “it’s the best in the city,” you know what city I mean because you read the sentence before it. An agent often only gets that one line, pulled out on its own, and it just breaks.

Some sites block agent traffic at the door. Usually that’s a 403, the flat no. Rate limiting throws a 429 to slow things down. Increasingly it’s a 402, Payment Required, which is Cloudflare’s pay-per-crawl asking the agent to open a wallet. And the door is closing fast, with Cloudflare Radar putting the all-crawler 4xx rate at roughly 14% in July 2025 and roughly 36% a year later. Some of that is deliberate policy, but a lot of it is bot mitigation nobody has revisited since 2019, quietly turning away agents you’d want to reach you.

• July 2025  —  14%
  July 2026  —  36%

Share of all crawler requests met with a 4xx rejection, per Cloudflare Radar.

So the agent shows up wanting to do one specific job, and what it usually gets handed is a page it can’t fully read, a form it can’t push the buttons on, or a door that’s locked.

• Legibility — owned by content
  Can the agent understand what’s here?
• Operability — owned by engineering
  Can the agent actually use it?

Two layers, two owners. Neither blocks the other, which is why you can start both this week.

Start with the facts on the page

For the marketing team and webmasters

• Sweep your pricing, packaging, and integration pages for anything stale. This is the highest-value thing on either list. A buyer’s agent doing vendor research will happily quote last year’s tier names or an integration you dropped, straight into somebody’s procurement doc, and not think twice about it.
• Rewrite headers and key claims so they hold up alone. No pronouns doing the work, no “it integrates with everything,” name the actual integration every time, since an agent often only sees that one line.
• Make sure case studies and proof points can be pulled out and used on their own. A lot of that content is written to be read start to finish, and an agent comparing you against two other vendors is going to grab one sentence, not the whole page.
• Optionally, publish an llms.txt. It’s a markdown file at the root of the site listing what you are and where your important pages live. I’m including it with a caveat, because this is the one everybody built and it mostly doesn’t work. Ahrefs looked at 137,000 domains and found 97% of llms.txt files got zero requests in a month. AI retrieval bots, the ones that actually decide whether you get cited, accounted for about 1% of the requests that did happen. The single largest consumer was SEO tools checking whether you have one. Google has said it does nothing for Search. It’s cheap, it may matter later, and it’s genuinely useful if you publish developer docs that coding agents read. Don’t lead with it.

Then make the page usable

For development

• Server-side render anything that carries a fact. Pricing, specs, integrations, availability. If a crawler can’t see it in the raw HTML, it doesn’t exist. This is the one to do first.
• Add schema.org structured data as JSON-LD on your product, pricing, FAQ and organization pages. This is the format machines were already built to read, it’s what Microsoft’s NLWeb runs on, and it’s the closest thing to a sure bet in this whole post.
• Fix your 404s and stop breaking URLs. In the Vercel data, ChatGPT’s crawler spent 34.8% of its fetches on 404s and Claude’s 34.2%, against 8.2% for Googlebot. You get a small slice of an agent’s attention and right now a third of it lands on nothing.
• Audit what your bot rules are actually blocking. Decide on purpose which agents you want in, allow them in robots.txt, and make sure a CAPTCHA isn’t stopping the ones you said yes to.
• Scope an MCP server against your two best tasks, not the whole site. Probably something like checking whether you support a specific integration, or pulling current pricing for a tier. Walk before you run.
• Get the API documented well enough that a machine could use it, not just whoever built it, since more of your buying committee’s research is going to run through an agent before it runs through a person.

Four things your team can do right now

1. Grep your server logs for GPTBot, ClaudeBot, OAI-SearchBot and PerplexityBot. Find out whether agents can even reach you, and what status codes they’re getting.
2. Load your pricing page with JavaScript disabled and look at what’s left.
3. Add JSON-LD to your top three fact pages.
4. Pull your top 404s and fix them.

The future is not set

One more correction to the standard pitch. People will tell you today’s content trains the models answering questions in 2029. Maybe, partly. The more urgent thing is that most AI answers right now are retrieved live, not recalled from training. ChatGPT Search, Perplexity and Gemini go fetch pages at the moment somebody asks and cite what they find. Which means the payoff isn’t years out, it’s whether you’re retrievable and quotable this week. That’s a better reason to move than the training argument, and it’s easier to check.

Every agent that leaves your site today is coming back tomorrow. “I’ll be back” only sounds like a threat if you weren’t ready for the robot uprising. Get the site in order and it’s just a bot doing its job.