Google has spent the last 10 years ago making Google worse. They achieved this in large part by making the whole Internet worse [0], but a search engine with results of the quality of Google 10 years ago would be a serious competitor.
[0] For example, Google used to have a fairly strictly enforced rule that indexable content had to actually be visible to an unauthenticated user. The current crop of sites that have apparently useful content in snippets but that hide it when loaded would have been penalized, possibly severely.
They really have a flywheel of internet destruction. The fact that they own the entire display advertising business + search and ping pong people from a 50% paid search listing to a CPM arbitrage SEO website and back is just gross.
I've been using DDG and Brave for a few years now, and I went back to Google yesterday because its the default for Chrome on my phone. I was startled at the difference in quality, especially with Brave vs Google. Brave typically prefers long-form writing and the quality of the articles is typically a lot higher than what I found using Google.
While I have many reasons that Google made the Internet worse (AMP, censoring search, forcing localized search results, privacy, etc.) I don't think the hidden content is their fault but rather that of publishers.
Publishers blamed Google for declining revenue since they had to make their content openly accessible and therefore free in order to be visible to users on search. The EU tried to make Facebook and Google pay publishers to account for this. I think allowing paywalled content was a compromise to prevent this legislation from passing.
That being said, I agree with the publishers especially since hypocritically Google and Facebook strictly don't allow scraping of their services and litigate those who do.
Google could easily fix this by putting a symbol or label on paywall-related content so you know not to click it.
After 2016, Google went on a crusade to save the universe by stamping out all misinformation and seems to have highly deranked all forums and blogs in favor of mainstream sites. This has made Google unusable for any political or controversial subject matter. This has also made their LLM efforts too cautious as they can't handle the political controversy of an LLM and can't verify that it will never return anything that offends anyone.
It does seem like Larry and Sergei are finally back trying to fix the excessively politically sensitive and overly cautious culture. Larry, having disappeared to Fiji for several years, must have been pretty bored or annoyed with running it.
Yesterday night I was trying to recall the name of a particular SCOTUS case that I had (semi-incorrectly) thought was connected to the 14th Amendment.
Bombed out on the SERPs two or three times, so I started looking for an very old article by Thomas Sowell that randomly introduced me to the case a few years back. I knew it was something regarding the unintended consequences of the Civil Rights Act.
Neither Google nor DDG gave me ANY useful results for 3-5 variations on “scholarly critique of the Civil Rights Act”. I eventually remembered the Sowell connection and even adding his name in quotes only got me to a page deboonking the article I was searching for!
I’m very far from being a right winger or whatever so I’ve never really experienced this sort of thing before, but my god is the “no no, you don’t REALLY want to search for THAT” silent censorship out of control. Millions to billions of people use these search engines daily and assume they’re mildly biased but otherwise shining portals to the sum total of human knowledge.
In hindsight, I suspect I would have immediately found the SCOTUS case with my initial search on 2010 era Google. Very much seems like my starter queries were triggering the Bad Think Detected algorithm.
[0] For example, Google used to have a fairly strictly enforced rule that indexable content had to actually be visible to an unauthenticated user. The current crop of sites that have apparently useful content in snippets but that hide it when loaded would have been penalized, possibly severely.