AI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.

Tea@programming.dev · edit-2 2 days ago

AI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.

katy ✨@lemmy.blahaj.zone · 2 days ago

wikipedia should install ai mazes on their servers

KumaSudosa@feddit.dk · 22 hours ago

Not in this case, to be fair. The only concern is cost - since Wiki wouldn’t be opposed to them getting their actual data - and AI mazes are designed to safeguard more sensitive data, not reducing cost

upandatom@lemmy.dbzer0.com · 20 hours ago

Nice analysis. Need more smart people like you in the world

KumaSudosa@feddit.dk · 20 hours ago

I agree with that assessment!

inbeesee@lemmy.world · 1 day ago

Are there alternatives besides Cloudflare’s solution?

skulblaka@sh.itjust.works · 23 hours ago

Nepenthes does about the same thing but isn’t managed by a corp.

brognak@lemm.ee · 16 hours ago

There’s also Anubis, but it uses proof of work not a maze.

https://anubis.techaro.lol/

AI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.

AI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.

How crawlers impact the operations of the Wikimedia projects – Diff