> Recently I also got "rate limited" after opening about three web pages.
People who haven’t used it logged out recently may be surprised to find that they have, for some time, made the site effectively unusable without an account. Doing one search and clicking a couple results gets you temporarily blocked. It’s effectively an account-required website now.
Just opened a private window to try this, I did one search and clicked on four results, then a second search and got a 429 error. That is wild. I guess it's an anti-scraper measure?
Given the occasional articles that crop up showing the sheer volume of badly-behaved (presumably) AI scraper bots this makes all kinds of sense.
I can't find it now, but sometime in the past week or so I saw something that (IIRC) related to the BBC (?) blocking a ton of badly-behaved obvious scraper traffic that was using Meta (?) user-agents but wasn't coming from the ASNs that Meta uses. The graphs looked like this ended up reducing their sustained traffic load by about 80%.
Items where I'm doubting my recall (since I didn't find anything relevant doing some quick searches) are marked with (?)
Weird. Maybe it just hates my last two ISPs (Google Fiber, Frontier).
The usual way I notice I'm not logged in is by getting blocked after interacting with ~3 different parts of the site within a minute. If I search, click a single repo, and stay in that repo without using search, it seems to go OK, but if I interact with search and then a couple repos, or search again, temp-banned.
I made a search index for github repo [1] because it takes quite some time for github to load the repositories page (which is the page to allow searching),
And sometimes even using the exact repo name in Google search, I cannot see the corresponding (non-popular) repo.
People who haven’t used it logged out recently may be surprised to find that they have, for some time, made the site effectively unusable without an account. Doing one search and clicking a couple results gets you temporarily blocked. It’s effectively an account-required website now.