I implemented a Virtual Private Cloud (VPC) on AWS with Elastic Container Service (ECS) using Terraform so that I could run Docker, and it ended up being about 2 orders of magnitude more expensive than Hetzner after all of the services were configured. For example, something as simple as AWS NAT Gateway costs $30 per month, and it can be challenging to get everything right if you don't use one. I haven't tried Heroku, but expect similar prices and nearly as high of a configuration burden.
So what's the point of all that? Seriously, one server with 16 cores and the memory bandwidth and storage speed of SSD drives should be able to serve many thousands of simultaneous requests and millions of users per month. I can't help but feel that the cloud infrastructure and microservice movement of the 2010s was.. a scam.
I just need a place to run Docker, similar to what we had with Linode for running a shell 20 years ago. And don't tell me how to set it all up. Just give me a turnkey Terraform (or Ansible-inspired declarative configuration management tool) setup that has stuff like load balancing and some degree of more advanced features like denial of service protection out of the box. What we used to think of as managed hosting, but open source, and with sane defaults for running standard suites like Laravel, Rails, etc.
I have to assume that I'm just terribly out of it and something like this already exists. Otherwise I can't understand why someone doesn't just offer this and provide a way for us to pay them the millions of dollars we lose to spinning our wheels on cloud infrastructure with 1% utilization.
> So what's the point of all that? Seriously, one server with 16 cores and the memory bandwidth and storage speed of SSD drives should be able to serve many thousands of simultaneous requests and millions of users per month. I can't help but feel that the cloud infrastructure and microservice movement of the 2010s was.. a scam.
You're just missing the whole point of why you need n>1 instances. You're focusing on what might be the least compelling reason for the majority of applications, which is scalability.
Reliability and fault tolerance is a major selling point, and you can't simply expect to have adequate performance in more than one region if you're providing your services out of a single box in a single data center. The laws of physics ensure your quality of service will suck.
Operations also compell you to have multiple instances, ranging from blue/green deployments to one-box deployment strategies. And are you going to allow your business to be down for significant amounts of time whenever you need to update your OS? How would you explain all that downtime to your boss/shareholders? Are the tens of dollars you save on infrastructure a good tradeoff?
Also, security is also a concern. You're better off isolating your backend from frontend services, and software-defined networks aren't a silver bullet.
And in the end, are the tradeoffs of not designing your web services the right way really worth it?
Agreed entirely for most. I have seen so many insanely complicated and eye wateringly expensive architectures for what are simple web apps on the big clouds. Often the team don't really understand the intricacies of all these services and how they interrelate which causes often cause a lot of stability problems.
Even if they don't, there is usually a huge tax on developer productivity with these architectures, which often outweighs any improved stability over the long term.
Let's keep in mind that modern hardware and software is very stable, generally.
Agreed - I worked for a company with 30 people dedicated entirely to managing the mess they'd created in AWS - and their AWS account manager somehow managed to keep convincing them to use more and newer AWS services to dig them in even further.
People talk about resume-driven development, which is definitely a factor, but there's also fear-driven development - you have a hunch the architecture could be a lot simpler and cheaper, but you don't want to make any criticism in case the others in your team think you are technically ignorant, inexperienced, out of touch etc.
>Let's keep in mind that modern hardware and software is very stable, generally
Not at any significant scale. dimm will fail, power will be down, disks will need replacement.
It is all about risks after all. If you are ok to have couple of hours of downtime if one of the memory sticks stops working - good for you. But generally any large enough business won't tolerate such risks.
I'm not saying that the cloud is the answer, but I don't see any future for single instance solution. And if you design your system like this, you are taking much more risks than necessary.
It's less insidious than that. Engineers like using AWS at startups because it benefits their future careers at larger companies. Startups also get $100k-$500k in free AWS credits. So there's no short term downside.
Ultimately it's the CTO/Technical Founders fault that this goes on. Avoid cloud services like the plague unless you really actually need them. They will slow you down and eat your wallet in the long run.
Cloud is about elasticity. At ${work} we spin up a ton of EC2 instances in anticipation of huge traffic spikes that happen predictably several times a day, in different regions, and spin them down afterwards.
It's also about simplicity. Many services come pre-configured on.AWS, with clustering, failover, backups, etc already present.
When you are small, you can sysadmin your server fine, and you don't yet need the cloud.
When you are colossal, you want everything custom, and the cloud would cost you a huge amount, so you go for your own servers (and possibly datacenters).
But for the midrange, the expense of hiring several more SREs to handle dedicated servers is usually higher than the entire AWS bill, and the cloud looks very reasonable.
I can’t help but think that a 10x user spike causing a 10x CPU spike is mostly caused by poorly performing web apps written in dynamic scripting languages and the ethos “Just add more servers, servers are cheaper than developers.”
Turns out, developers are cheaper than DevOps engineers and AWS budgets. So if a linear spike in servers necessitates a superlinear cost in operational complexity, maybe working highly performant web app backends is actually cheaper?
Counterpoint: for some years we had clients who happily lived on cheap as fuck plans but requested more at the anticipated calendar events (NY and the like).
Most of the time we just removed GHz limits, sometimes adding vCPUs in advance, they fared well.
Yes, if your elasticity requirements are not within hours but within weeks or months, adding and removing capacity can be done by traditional means, even by renting physical servers and putting them into your on-premises racks.
You missed the point - for some the 'elasticity' is just being able to consume 12-20GHz instead of ~1-2GHz 350 days of the year. Sure, you can deploy some advanced hyper-scaling arch to do so... or just move to a fatter instance.
Hybrid approaches are also really good, but I'm gonna be honest: If capital is not an issue and the growth is to be expected, I wouldn't steer into dedicated server region. It's just such an headache if it's not the core business
Dedicated servers are way less of a headache than cloud in my experience. Unless you are talking about buying and building your own hardware, which is a completely separate thing and not generally recommended. You can lease dedicated servers these days as easily as launching a cloud compute instance but way less than half the price. I pay around $265/mo for 32 cores and 128gb of ram with local, dual, mirrored 1tb SSD drives. Looks like a reserved instance with similar performance on AWS would be somewhere around $700/mo.
> I can't help but feel that the cloud infrastructure and microservice movement of the 2010s was.. a scam.
I think the general consensus is that the most-root cause for why cloud infrastructure and microservice architectures matter is more People, less Technology. Its hard to hire highly experienced people, especially at the hundreds-to-thousands of engineers scale that many companies operate at.
That, in isolation, is reasonable. The actually uncomfortable question that no one wants to address is: was that a scam. Put another way, how much of that was a zero-interest rate phenomena, and how much "state of art best way to engineer" things was influenced from that, and has root causes in US monetary policy.
Between AI and high interest rates: if you're not ready to spend the 2020s questioning every single thing you think you know about software engineering; you won't be long for the industry.
My thought on this is we've all been trained to follow the Google approach: thousands of machines, horizontally scalable, to handle any load. But this model was developed in 2004 when they wanted to hold the entire web in memory and could afford the enormous hardware outlay.
Computers are so powerful now this may be the wrong instinct. You can mind boggling amounts of processing for say... $30,000 in hardware, and even with redundancy it would be much simpler to manage.
> I think the general consensus is that the most-root cause for why cloud infrastructure and microservice architectures matter is more People, less Technology.
Indeed the main selling point of microservices is People, more specifically how to set hard boundaries on responsibilities and accountability while ensuring independent decision-making.
Another important selling point of microservices is flexibility. It's trivial to put together an entirely new service from scratch, written in whatever technology crosses your mind. With microservices you don't even need access to a repo.
> set hard boundaries on responsibilities and accountability while ensuring independent decision-making
This isn't really true as most of the time the services interact heavily and need to be aligned with each other not to break and work properly.
And all what you're speaking of can be achieved by properly modularizing the code of a monolithic application. And microservices if you want them can be run on a single machine, you don't need "the cloud" or multiple computers in general to run a bunch of REST apps if your single instance is powerful enough.
And for most web apps it is with the currently available hardware.
> This isn't really true as most of the time the services interact heavily and need to be aligned with each other not to break and work properly.
I'm not sure you got the point. The need to provide interfaces and establish contracts and pledge SLAs is irrelevant and a non-issue. Naturally, your goal is to have a perfect uptime and never break interfaces, just like your goal when working on a monolith is to not add broken code that crashes the service.
The whole point of microservices is that you are entirely free to make any decision regarding the things you own, and your only constraint is the interfaces you provide and consume, and SLAs. From those interfaces inward, you are totally free to make any decision you think is best without having to bother about anyone else's opinions and technical choices and tastes and permissions. You can run one service behind your interface or hundreds. As long as your interfaces are up, you can do whatever you like. That greatly simplifies your problem domain.
> And all what you're speaking of can be achieved by properly modularizing the code of a monolithic application.
No, no you can't. Think about it for a second: can you pick your software stack? Can you pick what dependencies you adopt? Can you rewrite your code in any other language? Etc etc etc. Your hands are tied left and right.
An even more important consideration is being able to release features (and even more importantly roll back) in isolation when the team is ready. The more teams contributing to a monolith, the harder this becomes. I am sure someone here has seen it done with a 10 million line monolith built by 100 geographically distributed teams, but I suspect that is exceedingly rare.
Microservices aren’t a silver bullet for this, of course. They can be part of a winning recipe though.
While picking any stack you want may sound wonderful to the individual developer, it's less so for the company as a whole that has to maintain different languages and frameworks, when that developer moves on and now they have to find or hire someone else to keep that service running.
> While picking any stack you want may sound wonderful to the individual developer, it's less so for the company as a whole that has to maintain different languages and frameworks
This is orthogonal issue.
You can pick any stack, but you don't have to. There still can be general company wide rules about what is acceptable and what is not. It's just that this is informed and enforced by the actual needs (not being too diverse in the technologies deployed), that can change in time independently from contents of any code repository.
Plus you can still make strategic choices here and there, that don't comply with general rules, and have, say 95% of teams working under stricter limits of what's allowed, and this couple of teams with special needs, where you were free to decide otherwise.
> While picking any stack you want may sound wonderful to the individual developer, it's less so for the company as a whole (...)
Nonsense. Each team owns it's services, and they know best what works and doesn't work. It makes no sense to claim organizational inertia as a valid reason to prevent you from picking the best tool for the job.
To add to that, microservices allow you to gradually upgrade services without having to be blocked by the idea of having to go through major rewrites. You just peel off a concern, get it up and running, and direct traffic to it. Done.
"I haven't tried Heroku, but expect similar prices and nearly as high of a configuration burden."
Are you kidding me? Prices, sure, but comparing the configuration burden using AWS or GCP alphabet soup to a dead-simple Heroku setup is ridiculous. You set up something like hirefire.io for autoscaling, you connect your Github repo and that's it. You're done. Compared to weeks worth of headaches setting up Terraform Cloud and CI actions and massaging your state file and spending days tearing your headout over insane obtuse NAT configuration parameters? It's literally night and day.
Ya I think you're right. I regret not trying Heroku first.
I'm glad to have learned AWS, although I don't know that I'll build another cloud server. I think the biggest problem I found is that services tend to require each other. So I ended up needing an IAM for everything, a security group for everything, a network setting for everything.. you get the idea. A la carte ends up being a fantasy. Because of that interdependency, I don't see how it would be possible to maintain a cloud server without Terraform. And if that's the case, then the services just becomes modules in a larger monolith. Which to me, looks like cloud servers are focusing on the wrong level of abstraction. Which is why preconfigured servers like Heroku exist, it sounds like.
An open source implementation of Heroku running on Terraform on AWS/GCP could be compelling. Also a server that emulates AWS/GCP services so that an existing Terraform setup could be ported straight to something like Hetzner or Linode with little modification. With a migration path to start dropping services, perhaps by running under a single identity with web server permissions (instead of superuser). And no security groups or network settings, just key-based authentication like how the web works. More like Tailscale, so remote services would appear local with little or no configuration.
Also this is a little off-topic, but I'd get rid of regions and have the hosting provider maintain edge servers internally using something like RAFT combined with an old-school cache like Varnish to emulate a CDN. The customer should be free from worrying about all static files, and only have to pay a little extra for ingress for the upload portion of HTTP requests for POST/PUT/PATCH and request headers and payloads.
Oh and the database should be self-scaling, as long as the customer uses deterministic columns, so no NOW() or RAND(), although it should still handle CURRENT_TIMESTAMP for created_at and updated_at columns so that Laravel/Rails work out of the box. So at least as good as rqlite, if we're dreaming!
Edit: I don't mean to be so hard on AWS here. I think the concept of sharing datacenter resources is absolutely brilliant. I just wish they offered a ~$30/mo server preconfigured with whatever's needed to run a basic WordPress site, for example.
The price disparity is getting worse. Cloud price reductions haven’t kept pace with the amazing pace of hardware performance improvements, especially in I/O and RAM quantity.
For $400 you can buy an SSD with 1.5 million sustained IoPS.
It is no big deal to build a 1TiB RAM box these days.
If it wasn't for a good portion of tech leads FAANG aspirational goals I bet a lot more companies would be on monoliths. It's simply staggering the productivity and cost differences (up to of course a certain point) being on monoliths vs microservices. The discussions on authorisation alone.....
> If it wasn't for a good portion of tech leads FAANG aspirational goals I bet a lot more companies would be on monoliths.
I think you inadvertently pointed out one of the advantages of microservices: the ability to bootstrap ideas without having to be hindered by artificial barriers to entry. If you have an idea and want to show it off, you just implement it and make it available to the public. Call it "aspirational goals", but I don't see any advantage in barring people from providing value without having to wrangle bureaucracies, red tape, and office politics.
If you ignore the costs from stuff like vmware, vcenter, windows server, etc... sure. But it's weird to forget that managing VMs was a huge expense even before the shift to the cloud. No one was just hosting a few dozens or a few hundreds VMs without very expensive management software.
I’m not saying that you shouldn’t use the cloud for deploying your software. But instead of deploying hundreds of small services, you may just deploy a handful bigger ones, and are still good. And you may also not need 30 different AWS services, you may be good with a database, maybe a message broker and a container runtime.
If you keep the infrastructure simple, it’s also much easier to switch cloud provider and to set up testing systems.
The cloud almost always ends up costing a lot more at scale and increasingly at the beginning.
In some part this is because the cloud is someone else’s computers that you have to help pay for.
The current incarnation of cloud was designed in part to work around scaling issues with Linux networking, which have long since been solved. What a $5 Linode/VPS can do today vs 10 years ago is much different from a linux standpoint alone before additional horsepower.
At the same time tooling to host (docker, ansible, terraform) is making traditional devops heavier at times.
I think one disadvantage of putting everything (nginx, web server, database, monitoring tools, etc.) in one machine is that suddenly your machine is exposing a myriad of ports to the internet and one mistake on your side (e.g., misconfigured auth module) is all what's needed to compromise your entire service.
Having some sort of vpc where you can safely put your db server that only listens to requests from servers within the same vpc (e.g., web server) sounds like good practice to me.
> I think one disadvantage of putting everything (nginx, web server, database, monitoring tools, etc.) in one machine is that suddenly your machine is exposing a myriad of ports to the internet and one mistake on your side (e.g., misconfigured auth module) is all what's needed to compromise your entire service.
All the Linux distributions I got to know use sensible defaults so that critical services don't bind to a public-facing interface / bind only to localhost, e.g. mariadb and mysql on Debian.
Besides that, Hetzner's "Robot" interface allows to configure which ports/IP addresses you allow access to your Hetzner server.
Hetzner is great but it's not a good choice for companies which really care.
Your data is not encrypted in rest.
Their security is not bad but someone can easily plugin whatever they like.
Try this at Google. Google not just has much stricter physical access control but also FULL control over the hardware. They know the firmware of the Mainboard. The server don't even unencrypt and start if you move them out of their Datacenter.
Google doesn't just has ingress egress they have undersea cable.
There is so much difference between hetzner and google it's not the same thing.
> Hetzner is great but it's not a good choice for companies which really care.
Your data is not encrypted in rest.
It's up to you to set up "encryption in rest": E.g setup LUKS on a Hetzner machine.
> Their security is not bad but someone can easily plugin whatever they like.
That's not how it is. Check e.g. this six-year old security video from Hetzner: Hetzner is great but it's not a good choice for companies which really care.
Your data is not encrypted in rest.
Their security is not bad but someone can easily plugin whatever they like.
I know the video quite well and I also have stuff on hetzner.
But it's just levels different to Google.
I have a small startup at hetzner.
But my main job has everything at the big cloud providers because of audit logs, high slash, global network etc. And no the hetzner video shows already that hetzner can't fullfil all required security certifications.
They have rows and rows of desktop PCs there.
With 'really' care I mean if you process millions or billions you just don't go to hetzner.
And doing encryption on rest with what decryption method? Manual unlock through a remote console?
At Google you literally have encryption on rest by default with key rotation build in and you can use your own keys.
So what's the point of all that? Seriously, one server with 16 cores and the memory bandwidth and storage speed of SSD drives should be able to serve many thousands of simultaneous requests and millions of users per month. I can't help but feel that the cloud infrastructure and microservice movement of the 2010s was.. a scam.
I just need a place to run Docker, similar to what we had with Linode for running a shell 20 years ago. And don't tell me how to set it all up. Just give me a turnkey Terraform (or Ansible-inspired declarative configuration management tool) setup that has stuff like load balancing and some degree of more advanced features like denial of service protection out of the box. What we used to think of as managed hosting, but open source, and with sane defaults for running standard suites like Laravel, Rails, etc.
I have to assume that I'm just terribly out of it and something like this already exists. Otherwise I can't understand why someone doesn't just offer this and provide a way for us to pay them the millions of dollars we lose to spinning our wheels on cloud infrastructure with 1% utilization.