Think of your website like a museum.
Visitors are welcome to explore the exhibits that are open to the public, but certain areas like staff offices, storage rooms, or maintenance areas are off limits. Instead of locking every door, the museum posts signs telling visitors where they should and shouldn’t go.
A robots.txt file works the same way for search engines. It tells web crawlers which parts of a website they can explore and which sections they should avoid.
Now, as AI-powered search and large language models (LLMs) become more common, publishers are looking for similar ways to guide AI systems. That’s where llms.txt comes in.
Rather than controlling what AI crawlers can access, llms.txt is designed to help AI platforms understand which parts of a website are most important. In simple terms, it acts more like a curated guidebook to the museum by highlighting the exhibits that matter most.
Straight North AI and Search SEO Expert Tom Lustina explains why that guidance is not as important.
Then llms.txt comes in and says, okay, we’re not telling you how to crawl the site, but we’re saying, this is what the site’s about, and these are the most important sections. So, it’s almost like a cheat sheet for crawling or having that information. A site like Google doesn’t need it. The other AI platforms are saying, we don’t need this. We can follow the crawl directive, we can crawl a site, we can figure that out for ourselves. So, it’s, like, it’s this cheat sheet that, at least at this point, isn’t needed.
Some publishers and SEO professionals are experimenting with it, however, major search engines and AI platforms have not adopted it as a formal standard, and its direct impact on visibility remains uncertain.
The bigger question isn’t whether llms.txt matters today. It’s why the industry is looking for it in the first place. As AI-generated answers become more common, publishers, brands, and content creators are increasingly concerned with how their content is interpreted, attributed, and surfaced. Understanding those concerns and where AI search is headed next is far more important than the file itself.
Why Publishers and Creators Care About AI Content Usage
Google has spent years refining how it delivers information in traditional search. When someone asks a question, Google crawls web pages, identifies the most relevant sources, and surfaces those results directly on the search page. The response it returns is quoted text. AI platforms work differently. Instead of quoting text, they pull information from across the web and combine it into a single synthesized response.
That shift creates new challenges for brands and publishers. Since AI-generated answers blend content from multiple sources, information can lose context, attribution can become unclear, and AI-generated summaries may unintentionally distort a brand’s intended message.
“The challenge for brands, for content creators, for writers, and even for comedians is the fear that a synthesized response is going to be attributed to their brand. It could be wrong, or it could clearly just be something that they’ve never said before, and they don’t want to be attributed to them.”
Some experts compare it to a “Frankenstein recipe,” which is an AI-generated answer built from pieces of different sources that may not accurately reflect any one creator’s original content. AI may combine multiple recipes into one flawed version, and the original brand could receive blame for inaccurate results.
Beyond attribution concerns, many publishers are also worried about how AI systems learn from and reuse content. If AI platforms can summarize proprietary information without sending users back to the original source, publishers lose traffic, visibility, and some of the competitive advantage that comes from creating unique content. As AI search evolves, content ownership and reuse policies are becoming increasingly important topics across the publishing industry.
Why Quoted Results vs. Synthesized Results Matters
Quoted results and synthesized results represent two different ways AI platforms can use website content. Quoted results rely on direct excerpts from a source, preserving the publisher’s original wording and allowing clearer attribution and control over how content appears. Synthesized results, by contrast, combine information from multiple sources into a new summary generated by the AI. Publishers may prefer quoted results because they provide transparency, maintain brand context, and reduce the need for restrictive directives, while platforms like Google appear to favor greater flexibility with synthesized results, believing they can independently determine how to use content in ways that best serve users.
Lustina explains this situation further with a comedian example:
You could also take, like, a comedian like Jerry Seinfeld, for example, which is one of my favorite examples to use, it helps their brand if you are asking about a joke that they’ve told in the past. Like, if Jerry Seinfeld has this great joke about, about reservations for a restaurant. You can think of the exact… the exact joke. Google or any platform can say, here, you know, here’s the joke that he told, and it’s given him credit for an exact joke that he told. If it were synthesized, that is opening up the potential for a brand new joke, a synthesized joke that is attributed to Jerry Seinfeld. Jerry Seinfeld gets paid a lot of money for creating his jokes. Nobody else should be creating a joke and claiming that it’s Jerry Seinfeld, which would be a synthesized result in that. It’s the same thing for the Frankenstein recipes. They want credit for their exact quoted text. They don’t want any platform to make up something new, and then give it, and then say, hey, this brand, or Jerry Seinfeld wrote this.
Currently, no major platform supports these controls, but proponents of llms.txt envision future scenarios where publishers could define how their content is used. For example, a publisher might allow AI systems to quote content directly while prohibiting synthesized responses, permit search indexing but restrict model training, or establish different permissions for different AI platforms.
The concern for publishers and brands is that AI systems may summarize, remix, or reinterpret content in ways that dilute the original message. In some cases, AI-generated responses may sound authoritative, but they fail to accurately represent the creator’s intent.
Part of the challenge is that search platforms increasingly have to decide when a direct quote is more appropriate than a synthesized answer. Lustina notes that Google’s long-term vision is not to rely on publisher-supplied directives to make that distinction. Instead, the company wants its systems to determine for themselves whether a user would be better served by an exact quoted response or a newly generated summary based on multiple sources. That philosophy helps explain why Google has shown little interest in adopting llms.txt as a formal standard.
Should Businesses Implement llms.txt Right Now?
At the moment, llms.txt remains an experimental concept rather than an established SEO standard. Google and other platforms have publicly stated that they do not use llms.txt. In fact, Google has publicly indicated that the file is not necessary. Some publishers see llms.txt as a way to provide AI systems with a shortcut to understanding their content, yet major platforms have not signaled that they need or intend to use that guidance.
Because of this, there is currently no proven SEO or GEO benefit to implementing llms.txt. Most experts view it as a low-priority initiative compared to foundational SEO practices like technical optimization, high-quality content, site structure, and E-E-A-T signals.
Advocates of llms.txt argue that implementation requires relatively little effort and carries minimal risk. For organizations interested in experimenting with emerging AI governance practices, adding the file may serve as a way to stay informed about evolving standards and prepare for future adoption if industry support grows.
Lustina recommends approaching llms.txt cautiously for now. Since standards have not yet been established and measurable value remains unclear, most businesses are better off monitoring developments rather than prioritizing implementation immediately. While adding llms.txt is unlikely to cause harm, it should currently be viewed as optional rather than essential.
Good SEO Is Still Good GEO
Traditional SEO remains the foundation of GEO. AI search experiences still rely heavily on traditional search results. When an AI agent searches for information on a topic, it may evaluate hundreds of websites before generating a synthesized response.
It definitely is, and that’s something that we say constantly. Good SEO is good GEO. The same things that benefit you from a Google traditional search perspective are those things that help you in all of the AI platforms. And the easiest connection to make, and it is an agentic behavior, but… AI platforms, if you ask, if you ask an informational question, their agentic behavior is to perform that question and similar questions within traditional search, come up with all of the pages that are in the results, and then consider those pages for the synthesized result. So the starting point of AI, or AEO, GEO, the starting point is truly really good SEO. You are in the game. You are in consideration if you’re showing up in traditional search for those synthesized results.
It comes down to helpful content and crawlability. Also, when someone does an AI search, they may get an overview with links to the stories where the information originated. They then click through to those sites to discover more information. The sites that are the most helpful and laid out nicely are more likely to get referenced and seen.
The Bigger Shift: Agentic AI Behavior
Much of the discussion around llms.txt focuses on content discovery, attribution, and AI-generated answers, however, Lustina believes the larger long-term story involves something else entirely: helping AI take action on behalf of users.
The next phase of AI may be less about helping platforms understand information and more about helping them complete actions on behalf of users.
This concept is commonly referred to as agentic AI. Instead of simply answering questions, AI agents can perform tasks such as comparing products, gathering quotes, filling out forms, making reservations, or even completing purchases.
For many consumers, this behavior already feels familiar. Insurance comparison platforms gather quotes from multiple providers. Travel aggregators compare hotel and flight prices across numerous websites. AI agents represent the next evolution of that experience, allowing users to delegate increasingly complex tasks to software.
According to Lustina, this is where Google appears to be focusing its attention.
“Google is saying, we don’t need help crawling, we don’t need help with our results, we’ll figure that out on our own. Google needs help understanding what actions AI can take on a site on behalf of users.”
In other words, Google’s public comments suggest that the company believes it can determine how content should be crawled, indexed, and surfaced without relying on publisher-supplied directives such as llms.txt. However, AI agents still need guidance when interacting with websites and completing actions for users.
Why This Matters for SEO
As AI-powered search continues to evolve, future optimization efforts may extend beyond content visibility and into action enablement.
Google has already begun introducing protocols designed to help AI systems understand what actions are available on a website and how those actions should be performed. As AI agents become more capable of acting on behalf of users, websites may need clearer ways to communicate tasks such as requesting quotes, completing forms, scheduling appointments, comparing products, or initiating transactions. Emerging frameworks like WebMCP reflect this shift toward making website actions more accessible and understandable to AI systems, potentially creating new considerations for technical SEO and AI readiness.
For marketers and site owners, that represents a significant shift. Traditional SEO has focused on helping search engines discover and understand content. Agentic optimization may focus on helping AI systems understand what users can actually do once they arrive.
Standards are still emerging, and Lustina believes this is the area worth watching most closely. The fundamentals of SEO remain unchanged — high-quality content, crawlability, authority, and user experience still drive visibility across both traditional and AI-powered search. The new opportunity lies in helping AI agents act on behalf of consumers.
As AI search evolves, the question may become less about how websites are crawled and more about how effectively they can support AI-driven actions.
AI Search Is Evolving Into AI Action
The next challenge for AI may not be understanding website content, rather, it may be understanding website actions. Google has repeatedly suggested that its systems can already crawl, interpret, and surface content without requiring extensive site-side guidance. Where additional structure may be needed is in helping AI agents understand what tasks they can perform on behalf of users.
That could include requesting quotes, completing forms, comparing products, scheduling appointments, making reservations, or eventually completing purchases. As AI-powered assistants become more capable of acting on behalf of users, websites will need clearer ways to communicate which actions are available and how those actions should be performed.
This represents a shift in focus from helping AI discover information to helping AI take action. Content visibility will remain important, yet businesses may eventually need to think about how their websites support agent-driven interactions and whether key actions can be easily understood and executed by AI systems acting on behalf of consumers.
Where I do see the potential usage is for, instructing… is for the agentic behavior and actions. So everything from a crawl and results perspective is already taken care of. What’s next is how… how can an agent act on a consumer’s behalf on my site? What are the actions? Identifying those actions is still needed and very valuable, and that’s something that’s in the works right now. Google has just introduced the protocol for that. So, I don’t think llms.txt will go anywhere, because Google and the AI platforms can handle everything from a crawl and results perspective, or at least they want to do that on their own. Where they need help is knowing how an AI bot can act on a user’s behalf on your site.
What SEOs Should Watch for Moving Forward
As AI-powered search experiences continue to evolve, the fundamentals of SEO remain as important as ever. High-quality content, strong site architecture, crawlability, and a positive user experience will continue to serve as the foundation for visibility across both traditional search and AI-driven interfaces. However, marketers should also begin paying attention to how AI agents interact with websites. Google’s recent comments suggest that future optimization efforts may not focus solely on helping AI systems understand content, but also on helping them understand the actions users can take. Whether that means requesting quotes, completing forms, booking services, or making purchases, the ability to clearly communicate available actions could become an increasingly important component of digital strategy.
For marketers, this represents a potentially significant shift in optimization priorities. Historically, SEO has focused on helping search engines discover, crawl, and understand content. In an agent-driven environment, businesses may also need to think about how clearly they communicate the actions users can take on their websites. The next generation of optimization may involve helping AI agents not only understand information, but also complete tasks.
So, there is a new part of the game to pay attention to. The thing that we always should focus on from an SEO and GEO perspective is how well is your site crawled, and how helpful is your content to your audience? That’s what gets you in results, whether it’s, whether it’s traditional search or an AI platform. The new level that’s coming out now, to think about in the future is how to help AI platforms act on a consumer’s behalf. So, and that’s still…coming out. That’s still something that remains to be seen, so I wouldn’t say that there’s an action right now for that, but that is something to look forward to, is making it easier for an AI platform to act for a user on your site.
Brand ownership and visibility may become even more valuable as AI systems increasingly act as intermediaries between businesses and consumers. While the exact standards are still taking shape, one thing is clear: the next frontier of optimization may involve preparing websites not only to be understood by AI, but also to be used by AI on behalf of users.
Key Takeaways
- Content ownership and attribution remain major concerns.
- Google is shifting from content understanding to action enablement.
- SEO remains the foundation of AI visibility.
- Agent-ready websites will gain an advantage.
- The next frontier is optimizing for AI-driven actions.
