minus-squarePlantPowerPhysicist@discuss.tchncs.detoOpen Source@lemmy.ml•The Open-Source Software Saving the Internet From AI Bot Scraperslinkfedilinkarrow-up32·5 days agothe scrapers ignore robots.txt. It doesn’t really ban them - it just asks them not to access things, but they are programmed by assholes. linkfedilink
PlantPowerPhysicist@discuss.tchncs.de to Linux@lemmy.ml · 6 months agoThank you Microsoft, for the final pushplus-squarediscuss.tchncs.deimagemessage-square0fedilinkarrow-up15arrow-down10
arrow-up15arrow-down1imageThank you Microsoft, for the final pushplus-squarediscuss.tchncs.dePlantPowerPhysicist@discuss.tchncs.de to Linux@lemmy.ml · 6 months agomessage-square0fedilink
the scrapers ignore robots.txt. It doesn’t really ban them - it just asks them not to access things, but they are programmed by assholes.