It makes sense if you are handling millions of requests from all over the world per second and need failovers if machines go down.
But...if you just want to run your own personal search engine say...
Then for Wikipedia/Stackoverflow/Quora size datasets (50GB with with 10GB worth of every kind of index(regex/geo/full text etc) ) you can run real time indexing on live updates with all the advanced search options you see under their "advanced search" page one any random Dell or HP Desktop with about 6-8GB of RAM.
Lots of people do this on Wall Street. People don't get what is possible on desktop cause so much of it has moved to the cloud. It will come back to desktop imho.
It won't come back to desktops because as they get cheaper the costs of people to maintain them increases.
There will always be people who need them and use them but that proportion is going to keep decreasing (I'm somewhat sad about this, but the math is hard to argue with).
Just temporary. Nature did not need to invent cloud computing to perform massive computation. The speed and data involved in computation going on in a cell or an ants brain show us how far we can still go on desktop.
The breakthroughs will come.
In the meantime (unless you are dealing with video) most text and image datasets out there that avg Joe needs can easily be stored/processed entirely locally thanks to cheap terabyte drives/multicore chips these days. People just haven't realized there isn't that much useable textual data OR that local computing doesn't require all the overhead of handling millions of requests a second. This is Google problem not an avg Joe problem that is being solved with cloud compute.
No idea what cost that comment refers too. If you are Facebook or YouTube or Twitter or Amazon sure you need your five thousand man PAID army of content reviewers to keep the data clean cause the data is mostly useless. But if you are running Wikipedia search or Stackoverflow's search or the team at the library of congress please go and take a look at the size of these teams and their budgets and their growth rates.
Are you conflating "single server" and "desktop computer" here? It sounds a lot like you might be?
Because almost everyone already uses the model of co-locating the search index and query code on a single computer (both Wikipedia and Stackoverflow use Elastic Search which does this).
They use multiple physical servers because of the number of simultaneous requests they serve.
This has never been the use-case for Hadoop.
I've built Hadoop based infrastructure for redundantly storing multiple PB of unstructured data with Spark on top for analysis. This is completely different to search.
That's very different to the Wall St analyst running desktop analysis in Matlab, or the oil/gas exploration team doing the same thing.
1. You assume stopping "revolution" is the goal. It is not. All you have to do is look at what outcomes "revolutions" in an info saturated/low attention span/consumption culture based society have produced in the last 20 years.
2. In 2013 the Washington post did an estimate on how much was being spent on the surveillance society in the US. Everyone reacted in disbelief. Do you know by how much that budget has increased in the last 5 years? Are these people just brain-dead to be spending this kind of cash? Ofcourse not.
The US has made its share of mistakes over valuing freedom and squandering potential. China is overvaluing control and will make its share of mistakes too. We have to learn from both sets of mistakes to arrive at the right balance of where society should go in high noise info saturation environments. These are very new environment that society hasnt been in before and the right path ahead is not as obvious as people think.
Who watches the watchers? Government bureaucrats are human too, and why are they somehow superior to the rest of us in making decisions? If you have a bee-watcher, you must then have a bee-watcher-watcher, an so forth. If everyone makes their own decisions, we call that freedom, and it means we don't have some fallible bureaucrat deciding what is best for me (which I very much doubt they will do well).
Just cheaper these days to buy them from India. You could fly there, get a top quality doc/audiologist to check you up. The same top of the line aids cost around 2-3K max. Do the touristy stuff and fly back all for less than what it costs to get an aid in the US. "Medical tourism" they call it.
If you are taking the trouble to look into how to save money on hearing aids, Costco will get you audiologist testing and hearing aids at <$2000 and vastly less trouble than a trip to India. (That's what I did for my last replacement pair last year and so far it's worked out well.)
This where I tell couples "it takes a village". In a healthy working "village" there is always someone around who will compensate for weaknesses in both partners that aren't easy to fix. Modern couples have this highly misguided notion that between themselves they can work their way through complex things they aren't trained to handle. The larger their support group (parents, in laws, family, kids, friends) the better the outcomes.
The fix is not complicated. I have no issue with my search data/post viewing history providing me better results or better ads.
The real damaging data to people and society at large is a different set of data. It's the publicly visible counts next to every thought and utterance reinforcing misguided beliefs and behaviour up and down the food chain constantly. Any experienced shrink, psycologist or educator, marketing/PR expert knows applying the right amount of feedback at the right time is critical to how people process info.
Remove/delay/reduce the visibility of like counts/view counts/upvotes/retweet counts that are displayed and the world will be a different place overnight.
>I have no issue with my search data/post viewing history providing me better results or better ads. The real damaging data to people and society at large is a different set of data.
Hmm, I disagree. It seems to me that both sources of data are dangerous. Yes, showing "You have 10 likes" is bad for users as social media companies iterate their way towards addiction but your search data is a toxic asset beyond just showing ads. Let's say google leaked everyone's search history tomorrow. How many marriages are going to be ruined? How many politicians are going to resign? How many future politicians will decide to never run? How many firings and never-hirings will there be based on that history?
People should be able to live normal lives without being surveilled - by governments or corporations.
What if the opposite happened? What if we all suddenly had undeniable confirmation that shitloads of people watch mannequin porn, and so fuck it, why be embarrassed by it?
It doesn't seem to work that way. When Ashley Madison leaked did people stand up and go "whoa millions of men are trying to cheat on their spouses, maybe we shouldn't be embarrassed about this"? No, they said "whoa there are millions of fucked up men."
What about when all those celebrity nudies leaked? Did people stand up and say "maybe taking and sending nude photos isn't something we should be embarrassed about"? No, they said "look at this crazy photo of <Celebrity>!"
Not necessarily. If literally everybody is open to the same level of scrutiny of their lives, how quick will they be to judge others, lest they in turn be judged?
I think its more likely that the judgement will spread in a highly non-uniform manner that reflected existing social biases with some people being vastly more negatively affected by having their secrets exposed than others.
Given equal access to the sources of shame of all three, would we really expect society to apply equal judgement standards to a mother, a billionaire asshole white guy, and a black guy from the inner city?
My interpretation of the argument was that taking privacy away from those who had it wouldn't necessarily cause harm because in some aggregated way everyone would have something to lose and so people would be tempered against 'throwing the first stone'.
I think that notion is false because the negative effects of the loss of privacy will spread unequally and opportunity created for those to exploit the unequal consequences against their foes.
Very quick, it turns out. Judging others is a deflective tactic against being judged yourself. "People in glass houses shouldn't throw stones" is great spiritual wisdom but most people don't abide by it.
That's sorta my point. If no one had any privacy, then any time you judge others you open yourself to being judged. It's like punching someone in the face, you could do it but they'll probably punch you back.
So yeah, maybe you could make a big deal about congressman so-and-so having a diaper fetish or whatever, but then people will probably want to look into your porn browsing history.
They've just been punched in the face, they can't reasonably punch back. (Everyone has a plan until they get punched in the mouth).
I know it seems like universal surveillance could level the field and make people less likely to judge, but in practice it doesn't work that way. You say "people will probably want into [the judger's] porn browsing history", but in reality, they don't. The preemptive strike generally wins, and counterstrikes generally look like defensive posturing. So removing all privacy just gives more power to the bigger asshole.
Just had a conversation about Amazon facial recognition with a friend last evening.
Imagine ten years in the future: you did something to upset the local police officer. Maybe you didn't pick up a can like he ordered. In any case, imagine he takes a photo of you with his body camera and then using Amazon's machine learning, they're able to find likeness of you doing "illegal" things and immediately write you a ticket. Resist more? Maybe Amazon can dig deeper and find out the faces you are around often and dig out treasure trove about someone.
I agree. Without universal enforcement, universal surveillance is pointless. This is why I was glad to see TSA selecting old women on wheelchairs with oxygen masks for searching while boarding onto airplanes because what we had before that was not random searching at all. It was profiling.
I'm amazed nobody in New York talks about NYPD reflective vests in a car's dash. Clearly, the car owner is communicating that the car belongs to a police officer to avoid a ticket. Anyone who does this does not belong in our police force. However, people just don't care about it.
Broadly speaking, prejudice can be and often is (much) stronger than family ties, even if the perceived misdeed does not impact the family.
How well do you expect things to go if you move these sorts of situations into a context without any kind of strong, positive emotional bond?
A liberal society works not because things are allowed, but because people don't know. (And are taught to not stick their noise into other people's business).
Not many, considering marriages are less popular among young people nowadays.
> How many politicians are going to resign?
Not many, considering older politicians maybe don't spent their time on internet, and younger politicians are aware of the state of privacy on internet
> How many future politicians will decide to never run?
Maybe we will discover than many more people are "cleaner" than we thought, and will become influencers
> How many firings and never-hirings will there be based on that history?
That's a good point. However, the mega-corporation isn't 100% of the employment. Small enterprises and freelancer just want the work to be done and get pay, without wondering about who did what in their lives.
As another comment said, maybe watching porn and being interested in more than one partner isn't that bad after all. It only shows the dumbness of the artificial rules older authorities (such as religion, army, states, etc) imposed on crowd to keep control on them.
Edit : anyway, I support the idea of a private internet space for everyone, where people can express their ideas and experience with new concepts without fearing retribution. Words don't kill people.
I think that this is the solution. Remove the count and who of the likes. You can still show the fact that someone liked it for some stickiness but you remove race to post only to generate likes but rather to share a point of view.
I don't know about trust being the root. But the argument can be made that society is very different. And the Fred Rogers model won't have much of an impact given the environment of hyper stimulation/24x7 distraction that consumption culture creates.
Fred Rogers was critical of it even back then. Since then, consumption culture has exploded, objectification of women, violence, escapism whether it is in games, drugs or your social media, news streams is at an all time high. For a Fred Rogers type message to get through the volume dial on all this other stuff has to be dialed down. Otherwise the very valuable message gets lost in the noise.
I'd be curious to read any studies on child psychology with respect to how children would respond to something like Fred Rogers today. As in, would a child today be disinterested in the slow pace of that format and want to consume other media?
Interesting data there probably for a social psychologists/sociologist to dig into. More interesting than the amount of cheaters in a population or whether thats a fixed amount is how many go on to win and influence gaming culture (both the player and the designer).
Once upon a time I could find blog posts of people reviewing things, travel, software, equipment etc. Now I just find pages and pages of review aggregators. The advantage of finding a blog was didn't take more than a few clicks to figure out if the blogger and me shared thinking, lifestyles, financial means etc. Now all that is gone. We just have ridiculous aggregation.
The problem is exacerbated by search quality guidelines that need to be followed if you're going to rank for any competitive search term, especially when there are clear incentives for doing so - such as site speed and deploying ssl.
Coming next is the 'insecure site' warning that's likely going to cull many more hobby sites.
I can, because I just converted a site to https, which had a combination of relative and absolute href and src parameters on it in a confused manner. This was combined with html embedded in table records.
It wasn't a simple search and replace task since several admin functionalities were also included in the site.
Think about all of the sites that have relevant information that aren't selling anything that would love to get things like this fixed, but just can't throw money at the problem.
Before you say oh it shouldn't cost that much. Think about where some of these sites are hosted and for how little. Some people don't even have the means to pay for such a fix, but they have important information to share.
This probably should have been a rule applied to sites that sell things and not sites that just have information.
I am all for making money, but I also feel like doing some charity work for this problem as well. I heard the deadline is the 24th. Anyone else interested in doing this collectively for a couple sites?
> I can, because I just converted a site to https, which had a combination of relative and absolute href and src parameters on it in a confused manner. This was combined with html embedded in table records.
Just add an upgrade-insecure-requests header to your webserver config, boom. No search and replacing needed.
> Most notably, mixed content checking [MIX] has the potential to cause real headache for administrators tasked with moving substantial amounts of legacy content onto HTTPS. In particular, going through old content and rewriting resource URLs manually is a huge undertaking.
I mean, uh sure, I'll volunteer to move your sites to https, but I don't think giving a random dude on the internet root access to fix the webserver config is a good idea ;-)
This site also had mixed content due to it using a forward proxy with ARR in IIS. Since ARR doesn't forward https requests it is truly turning into a mess. There isn't an option to just move it to another webserver as that would be it's own undertaking with the dynamic part of the site being ColdFusion.
> I mean, uh sure, I'll volunteer to move your sites to https, but I don't think giving a random dude on the internet root access to fix the webserver config is a good idea ;-)
In this point yeah I agree you don't want to let just random dudes have root access to a site. On the other hand I run my own legitimate consulting business and if you think about it. Every time I am winning over a new client for all intents and purposes I am just a random dude. :)
You can always just put an nginx in front of it to terminate TLS there. Sounds like a legacy mess nobody intends to maintain anymore anyways. Or hell, cloudflare them. Kinda pointless since you won't have TLS to the backend server, but the easiest "solution". I maintain that's it's possible to inject a header and terminate TLS however messy the system is within an hour.
Problem is that there's a ton of money in the "review industry," (through referrals) it's very lucrative, so the big ones are spending a lot to rank high.
Much more than just personal privacy. When CEOs, politicians, judges and generals use the internet too do you really want to be the guy/a company that gives them that call? The incentives are all messed up.
Yup good move. TV news orgs need to do this too. Next too all their flashing banners and scrolls. Especially when the talking heads of both side are raging. I'd really like to know if Coke or Apple or whoever is footing the bill.
But...if you just want to run your own personal search engine say...
Then for Wikipedia/Stackoverflow/Quora size datasets (50GB with with 10GB worth of every kind of index(regex/geo/full text etc) ) you can run real time indexing on live updates with all the advanced search options you see under their "advanced search" page one any random Dell or HP Desktop with about 6-8GB of RAM.
Lots of people do this on Wall Street. People don't get what is possible on desktop cause so much of it has moved to the cloud. It will come back to desktop imho.