Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> Recently I also got "rate limited" after opening about three web pages.

People who haven’t used it logged out recently may be surprised to find that they have, for some time, made the site effectively unusable without an account. Doing one search and clicking a couple results gets you temporarily blocked. It’s effectively an account-required website now.



Just opened a private window to try this, I did one search and clicked on four results, then a second search and got a 429 error. That is wild. I guess it's an anti-scraper measure?


Given the occasional articles that crop up showing the sheer volume of badly-behaved (presumably) AI scraper bots this makes all kinds of sense.

I can't find it now, but sometime in the past week or so I saw something that (IIRC) related to the BBC (?) blocking a ton of badly-behaved obvious scraper traffic that was using Meta (?) user-agents but wasn't coming from the ASNs that Meta uses. The graphs looked like this ended up reducing their sustained traffic load by about 80%.

Items where I'm doubting my recall (since I didn't find anything relevant doing some quick searches) are marked with (?)


I was pretty flabbergasted when I realized that if not logged in, I could no longer search a repository for code references.

You're really going to make me clone a project locally to do a search. I just end up using google to search github. It's so stupid.


Or, log in?


There is a huge irony in me logging in to this website to posit the question to you: why?


Thanks. I didn’t realize that. Migrating repos tonight.


They no longer allow sorting by number of stars in the search without being logged in either.


Stars can be bought, they aren't very reliable.


I have never noticed that and am rarely logged in.


Weird. Maybe it just hates my last two ISPs (Google Fiber, Frontier).

The usual way I notice I'm not logged in is by getting blocked after interacting with ~3 different parts of the site within a minute. If I search, click a single repo, and stay in that repo without using search, it seems to go OK, but if I interact with search and then a couple repos, or search again, temp-banned.


I made a search index for github repo [1] because it takes quite some time for github to load the repositories page (which is the page to allow searching),

And sometimes even using the exact repo name in Google search, I cannot see the corresponding (non-popular) repo.

[1] https://res-index.hkit.cc




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: