Crawlers and technical
How does OAI-SearchBot work?
OAI-SearchBot works as a web crawler associated with search retrieval, rather than the training-focused role of GPTBot.
Its user-agent identifies a particular crawling purpose. Site operators can express access preferences for that crawler through robots.txt.
Search access and answer selection are separate. An accessible page may still be omitted, and permission does not guarantee a citation.
In Seenford
For Seenford, this means separating technical access from observed visibility: every verdict is supported by an exact-sentence receipt.
Example
A public help page permits search crawling while using separate rules for training access.
Related terms
Updated
See what AI says about your brand →