Sekretaerbaer-Netz/Blocked_Bots_and_Crawler_Sekretaerbaer.de
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

add QualifiedBot

Browse source
This commit is contained in:
oldkid 2025-05-18 07:29:05 +02:00
parent ca3882e587
commit e67d06107b
1 changed files with 1 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
robots.txt
View file

@ -46,6 +46,7 @@ User-agent: PanguBot
User-agent: Perplexity-User
User-agent: PerplexityBot
User-agent: PetalBot
User-agent: QualifiedBot
User-agent: Scrapy
User-agent: SemrushBot-OCOB
User-agent: SemrushBot-SWA