Sekretaerbaer-Netz/Blocked_Bots_and_Crawler_Sekretaerbaer.de
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Bots and Crawler aktualisiert

Browse source 
add :
AI2Bot
Ai2Bot-Dolma
Brightbot
CCBot
ChatGPT-User
cohere-ai
cohere-training-data-crawler
Crawlspace
Diffbot
FriendlyCrawler
Google-Extended
GoogleOther
GoogleOther-Image
GoogleOther-Video
iaskspider
ICC-Crawler
ImagesiftBot
img2dataset
ISSCyberRiskCrawler
KangarooBot
OAI-SearchBot
omgili
omgilibot
PanguBot
Perplexity-User
PetalBot
Scrapy
SemrushBot-OCOB
SemrushBot-SWA
Sidetradeindexerbot
Timpibot
VelenPublicWebCrawler
Webzio-Extended
YouBot
This commit is contained in:
oldkid 2025-03-30 08:05:31 +00:00
parent 12c65f8188
commit 7567dd74f3
1 changed files with 34 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

35
Bots and Crawler
View file

@ -30,7 +30,6 @@ Meta-ExternalFetcher
FacebookBot FacebookBot
DuckAssistBot DuckAssistBot
Anthropic-ai Anthropic-ai
Bytespider
Bytedance Bytedance
fidget-spinner-bot fidget-spinner-bot
EmailCollector EmailCollector
@ -41,3 +40,37 @@ facebookcatalog
YandexBot YandexBot
Bingbot Bingbot
Googlebot Googlebot
AI2Bot
Ai2Bot-Dolma
Brightbot
CCBot
ChatGPT-User
cohere-ai
cohere-training-data-crawler
Crawlspace
Diffbot
FriendlyCrawler
Google-Extended
GoogleOther
GoogleOther-Image
GoogleOther-Video
iaskspider
ICC-Crawler
ImagesiftBot
img2dataset
ISSCyberRiskCrawler
KangarooBot
OAI-SearchBot
omgili
omgilibot
PanguBot
Perplexity-User
PetalBot
Scrapy
SemrushBot-OCOB
SemrushBot-SWA
Sidetradeindexerbot
Timpibot
VelenPublicWebCrawler
Webzio-Extended
YouBot