diff --git a/Bots and Crawler b/Bots and Crawler new file mode 100644 index 0000000..39b327f --- /dev/null +++ b/Bots and Crawler @@ -0,0 +1,41 @@ +Livelapbot +Franck, the Fediverse Graph Crawler +YisouSpider +DotBot +VelenPublicWebCrawler +ChatGPT-User +GoogleOther +GoogleOther-Image +GoogleOther-Video +Applebot +Applebot-Extended +AhrefsBot +Baiduspider +SemrushBot +DotBot +MJ12bot +PetalBot +SeznamBot +Mediapartners-Google +Bytespider +Claude-Web +GPTBot +PerplexityBot +Applebot +Amazonbot +Meta-ExternalAgent +Meta-ExternalFetcher +FacebookBot +DuckAssistBot +Anthropic-ai +Bytespider +Bytedance +fidget-spinner-bot +EmailCollector +WebEMailExtrac +ClaudeBot +facebookexternalhit +facebookcatalog +YandexBot +Bingbot +Googlebot