AI Search & GEO / AEO

llms.txt

llms.txt is an emerging proposed convention that places summaries and links to important pages in a Markdown file at the site root. It is intended to help AI crawlers or agents understand key information without exploring an entire site indiscriminately. Official support from Google or OpenAI has not been confirmed in the source material, although adoption by some tools and communities has encouraged early implementation. Unlike robots.txt, which governs access permissions, llms.txt provides a complementary summary of what matters on a site.

Business applications and impact#

Can support more efficient consumption of a site’s most important pages with fewer tokens.

Implementation guidance and considerations#

Publish summaries and links to important pages in Markdown at /llms.txt.

Crawling

크롤링

The process through which search crawlers follow links, discover URLs and retrieve new or updated content.

AI Crawler

AI 크롤러

A dedicated bot that collects web content for generative AI training data or real-time search-based answers.

Large Language Model

LLM

A deep-learning neural network pretrained on extensive text data to understand and generate language in a humanlike form.

Token

토큰

The smallest unit of text processed by an AI model, which may be a word, character or subword fragment.

Answer Engine Optimization

AEO

A strategy for optimizing content so that brands and their content can be cited in the single answers generated by AI search systems or voice assistants.