User-agent: * Allow: / # AI / dev crawlers — explicitly welcomed (we want citation in AI Overviews, # Perplexity answers, ChatGPT search, Claude.ai cite blocks, etc.). User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: PerplexityBot Allow: / User-agent: Applebot-Extended Allow: / User-agent: CCBot Allow: / User-agent: Google-Extended Allow: / User-agent: Amazonbot Allow: / # Webhook endpoint is not useful to crawlers. User-agent: * Disallow: /api/ Sitemap: https://opensource.yoda.digital/sitemap-index.xml