From 7567dd74f3909f61d70cde271457712875c945de Mon Sep 17 00:00:00 2001 From: oldkid Date: Sun, 30 Mar 2025 08:05:31 +0000 Subject: [PATCH] Bots and Crawler aktualisiert add : AI2Bot Ai2Bot-Dolma Brightbot CCBot ChatGPT-User cohere-ai cohere-training-data-crawler Crawlspace Diffbot FriendlyCrawler Google-Extended GoogleOther GoogleOther-Image GoogleOther-Video iaskspider ICC-Crawler ImagesiftBot img2dataset ISSCyberRiskCrawler KangarooBot OAI-SearchBot omgili omgilibot PanguBot Perplexity-User PetalBot Scrapy SemrushBot-OCOB SemrushBot-SWA Sidetradeindexerbot Timpibot VelenPublicWebCrawler Webzio-Extended YouBot --- Bots and Crawler | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/Bots and Crawler b/Bots and Crawler index cb14b5d..2cedcf2 100644 --- a/Bots and Crawler +++ b/Bots and Crawler @@ -30,7 +30,6 @@ Meta-ExternalFetcher FacebookBot DuckAssistBot Anthropic-ai -Bytespider Bytedance fidget-spinner-bot EmailCollector @@ -41,3 +40,37 @@ facebookcatalog YandexBot Bingbot Googlebot +AI2Bot +Ai2Bot-Dolma +Brightbot +CCBot +ChatGPT-User +cohere-ai +cohere-training-data-crawler +Crawlspace +Diffbot +FriendlyCrawler +Google-Extended +GoogleOther +GoogleOther-Image +GoogleOther-Video +iaskspider +ICC-Crawler +ImagesiftBot +img2dataset +ISSCyberRiskCrawler +KangarooBot +OAI-SearchBot +omgili +omgilibot +PanguBot +Perplexity-User +PetalBot +Scrapy +SemrushBot-OCOB +SemrushBot-SWA +Sidetradeindexerbot +Timpibot +VelenPublicWebCrawler +Webzio-Extended +YouBot \ No newline at end of file