Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Note the last image of the article is not correct, if you want full redundancy you need two independent power feeds. How independent depends on your needs of course. You might just want to run it from a different circuit but the same utility power. If you've got insane needs and megabucks you can talk to the utility about being fed from two substations or at the extreme end you can get one feed from the utility and you can make your own second feed. Traditionally we did this with water turbine working from a river but today I might look into solar and perhaps molten salt.

As an example, way back then when this was a very lucrative business, we placed the servers for a premium number erotic call in an industrial park on the border of two districts in Budapest because that's where we could get two independent power feeds without running our own lines. Internet connection wise, one was a simple leased line the other was a microwave connection to very far away. Short of bombing the entire site it was fair impossible for the installation to go offline and -- for the six years I knew about it, it never did. Note the site served German callers, that's where the big bucks came from.



>run it from a different circuit but the same utility power.

If you're going to come from a different circuit, see if you can at least pull it from a different phase, if we're talking a 120v circuit on 240v service (typical US home service). It's a small improvement, but I'd say 10%+ of the power outages I've seen have been just a single phase going out.


The center tapped transformer that provides residential split phase 120/240 on its secondary winding connects to just 1 of the 3 grid phases on its primary winding. If that one phase goes out on the grid, both sides of the split phase service go out together. There would need to be a fault on just one side of the split, downstream of that transformer, for your suggestion to hold up. Certainly not impossible, but far less common than a "losing one phase" scenario which would typically originate upstream on the high voltage side.

On a commercial 3 phase service, yes, connect redundant PSUs to separate phases, since each phase on the panel actually corresponds to each phase of the grid.


> The center tapped transformer that provides residential split phase 120/240 on its secondary winding connects to just 1 of the 3 grid phases on its primary winding.

I believe that's a USA peculiarity. Where I live, the usual residential and commercial wiring is from 13.8 kV to 127V/220V through a three-phase delta-wye transformer, in which the primary connects between each pair of phases, and the secondary connects between one phase and the neutral (the high-voltage primary side does not have a neutral). When one phase of the high voltage side is lost (very common, since each high-voltage phase is a separate wire and has an independent fuse upstream of the transformer), what happens is that one phase of the low voltage side stays normal (the one between the two intact high voltage phases), and the other two have a lower voltage which varies depending on their relative load.


> I believe that's a USA peculiarity.

Yep. Sounds like you have true 3-phase service, whereas most places in the US just get split-phase.


Is there a ELIA5 of what this distinction is?


Look up Wye vs Delta; but the basic point is in delta the phases are connected in series with each other, and in wye they are connected in parallel. It's easier to visualize when you can see the tap points on the transformer secondaries though.


You can definitely loose one phase of a split phase transformer. I've seen it 3 times in my life. Sometimes it is completely out, other times I've seen only getting 80v rather than the full 120.

All depends on how it failed.


But if the issue is a local breaker flip, then being on a different phase is very effective!

I don't have a power outage more than once a year, but we manage to blow a breaker more than a few times a decade (vacuum + water boiler was one).


If the issue is a breaker flip, then you only need to be on different breakers, right? Phase is irrelevant in that case


Right, because the main breaker would be 2 pole (or at the very least handle tied).


Generally, yes, but there are some exceptions like different breakers on a shared neutral, which may be tied so that both circuits break.


Agreed, as I said it seems like around a 10% improvement, which considering it's fairly easy to ensure the circuits are on different phases, is worth doing IMHO.


I have also seen a single phase go out (bird poop into a transformer) and having redundant PSUs on different phases saved our bacon


The house I grew up in lost transformers multiple times a year from squirrels.


I'm working on a facility with redundant power and my impression is that it's not insanely expensive if you have expensive machinery to protect and that diesel generators are far and away the most common and inexpensive second feed.


Worked at a place with a natural gas genny as the second source. You don't have to feed it, it just keeps working.


Technically you are just feeding it power from a different utility company than the electric company.


I wonder if natural gas is actually redundant at this point. Around here, phone company decided to take a dependency on the power grid, so phone + internet go down if the power is down.

It wouldn't surprise me at all if some utilities have started installing smart meter upgrades or inline compressors or computer-controlled valves that don't have generators attached to them.


> phone company decided to take a dependency on the power grid, so phone + internet go down if the power is down

I hope it goes down a couple hours later?

Around here (Germany) your phone and internet is also dependent on a junction box with routers somewhere within half a mile or so of your home having power. But they have four hours of battery backup, and on normal-sized outages they send people out with diesel generators when the batteries start running low (prioritizing business customers). Having it go down from power loss is a decision made in triage, not something that just happens


How common are power outages? Historically and in the coming years given nuclear power I hear is dead in Germany and France's reactors are old or need to shut down because of rivers being too hot in summer. Is the 4 hours a 1980s decision that needs revising?

Asking, coz Canada here. Power outages aren't uncommon (when I say that I mean: expect at least one that makes you get out the generator in 'shoulder season' per year), coz power lines (not talking transmission lines) are mostly above ground, except in large cities of course. But as soon as you get out of the "center" (which depending on city is larger or smaller too) it's good old wooden poles that carry power on the top and cable / phone on the lower level.

Bucket transformer[1] on a pole near you blows up is a favourite but the lines are actually fine, including your cable / DSL. Last time we also lost internet it took about 24 hours after power went out that internet went down as well. Cell phone service from the same company was still fine. The entire metro area was out of power for days and I guess they prioritized topping up the diesel/LP for those.

[1] These guys https://en.wikipedia.org/wiki/Distribution_transformer


Large-scale power outages are exceedingly uncommon in Germany, or really all of Europe. The last big one was in 2006 because of some mistakes when taking down a major transmission line [1]. I don't remember reactors being offline ever causing a power outage. When France has to shut down its nuclear reactors Germany just fires up more coal plants, that's the beauty of a large interconnected grid.

What does happen are smaller scale outages. Power lines are mostly buried along streets and under the sidewalk, just like telephone lines. That doesn't stop the occasional excavator digging too deep and taking a street off the grid. At an individual level it's extremely uncommon, maybe once a decade. But deploy thousands of boxes with networking equipment all around the country and it happens to your equipment all the time.

1: https://en.wikipedia.org/wiki/2006_European_blackout


Yeah historically that makes a lot of sense to me. Reactors going offline directly would usually be planned and thus not cause instability.

The 2006 one I had read about before. I love reading timelines of such disasters. Shows how hard this actually is and how much work it is to keep it all running.

Here's another one: https://en.wikipedia.org/wiki/Northeast_blackout_of_2003#Tim... And speaking of Canada and power lines (this time it does include transmission lines) https://en.wikipedia.org/wiki/January_1998_North_American_ic... While not so severe this is basically the kind of thing I was referring to us happening in "shoulder season". There's usually at least one ice storm or very wet snow event at the start and/or end of winter now and it's very likely that in our wooded area we get trees into power lines and boom the buckets go. When we're lucky it's localized and crews are available to come out and fix it in a few hours. If it's all over the place then it's gonna take a while and they'll have crews from other provinces and the US come in to help as well.

I'd be interested in your outlook on the future of the grid in Germany and Europe though. Of course when France takes a nuke offline, that's usually planned, even when done for a "river water temperature emergency" it's gonna take a while and you can bring that coal plant online like you mention. But doesn't Germany want to reach the climate goals it set itself? How does coal make sense there? And how is shutting down their own nukes a thing when it's OK to use French nuke power?

Or some natural gas, which is quicker. If you have the gas. Re: Russia.

In all of the European (NATO) countries together, is there enough generation capacity if you assume zero Russian inputs (save for say untraceable third party transit or resources) and half of France's nukes going offline? Especially when the sun doesn't shine because bad weather and thus the winds are so high that you have to shut down your wind turbines?


> There's usually at least one ice storm or very wet snow event at the start and/or end of winter now and it's very likely that in our wooded area we get trees into power lines and boom the buckets go

That's where a large, sparsely populated country is a real disadvantage. "Trees into powerlines" isn't really an event in Germany, since high voltage lines are 150 feet up in the air and kept clear of trees (they'll just cut a line though a forest for them). And everything smaller is generally buried. But that would be very hard to do in Canada. And of course we don't have to fight with ice on our transmission lines.

> How does coal make sense there?

Lobbying. And saving the jobs of hard-working coal miners is more romantic and appealing in election campaigns than saving the jobs of wind turbine manufacturers.

It used to make economic sense in the sense that coal plants were cheaper to run, but that has changed in recent years so what you see now is mostly inertia

> And how is shutting down their own nukes a thing when it's OK to use French nuke power?

Oh, there are lots of protests against French nuclear plants too, especially those in the border regions. We just can't do much about them. But the people who don't like nuclear plants aren't the ones running the energy markets.

On the future: Before 2022 the idea was to transition all coal capacity to gas. This was mostly happening on its own anyways due to gas outcompeting coal on price, and new pipelines like Nordstream were going to accelerate that economic pressure to transition. The Ukraine war was a big setback for that.

In the end I believe we are still moving to a future where a lot of power is coming from solar and offshore wind, with natural gas peaker plants to offset times without wind until grid-scale battery technology moves a bit along (molten salt, hot sand, pumped hydro in abandoned mines, etc). In addition to that obviously hydro and pumped hydro from the Scandinavian countries

We are far enough into economies of scale that the generation side is mostly going to sort itself out on economics alone. Solar is becoming dirt cheap, offshore wind is becoming profitable, natural gas is cleaner and cheaper than coal. The bigger issue are transmission lines. Building transmission lines takes decades because every NIMBY fights against them. But the existing transmission lines are built around a somewhat even spread of supply and demand, versus the new situation where we want offshore farms in the North Sea to be able to supply lots of electricity to the South when there's good wind, and the solar panels in the South to help power the North. And politicians from certain parties love to side with "their" NIMBYs for easy political points


In small town USA, my cable internet goes offline immediately even if the power outage is just a 1 second flicker. Then takes some equipment to reboot at the cable office before it comes back online. Very annoying. Phone companies typically have a good battery backup though.


At least during the Texas grid failures during their freeze the other year, gas ended up not being a backup because the lines froze over.

Something is only really a backup if the actual fuel is on site, at this point


Diesel generators are great if you need a few hours of backup (assuming the generator actually starts when you call on it).

But if you need enough backup capacity to survive something a multi-state, multi-day blackout [1] that probably gets expensive.

You wouldn't need that for a premium erotic call processor, but a 911 call exchange might, for the portion of their workload they can't pass off to another exchange.

[1] https://en.wikipedia.org/wiki/Northeast_blackout_of_2003


You can store about a day's worth of diesel on-site and have agreements in place to have daily refills in cases of emergencies.

If you can't get gasoline within a day's drive then there's bigger problems in the world.


In many places going over what you can store might put you over emission quotas and you would have to shut down anyway. I'm familiar with one incident at large DC which had fuel left and could easily get more, but only had few hours before they were required to shut down by EPA.


I worked at a site where we powered a bunch of stuff consistently from diesel generators until the grid hookup was finished, much longer than a few days. Probably was expensive.


Natural gas generators.


Looking at multi fuel generators is worth it too depending on your setup.

For example some can run gas/propane/natural gas


> for the six years I knew about it, it never did.

my dude i cannot tell you how much i love these stories of non-faang, real world engineering for extreme reliability.

thank you for posting that.


my fav real world engineering story is of the first commercial ISP in Hungary

back then running leased lines would've been way too expensive. So what did the kids running the show did? They got wind of a central office in an older part of Budapest have excess capacity so they rented an apartment in the next building and drilled the wall :D no expensive trenching, no expensive equipment to demultiplex landlines, nothing, just a bunch of wires running straight from the CO equipment into retail modems... We had no idea what we were doing, mind you. I was already doing Linux at the time so among the few installers I was the lucky guy who got to install the Internet at a small business who wanted it to be done on a Solaris workstation. That was a fun challenge... Other installs were Trumpet Winsock. The ISP itself ran a custom linux app, you dialed in and landed on a text app or maybe it was Lynx? can't quite remember, it's been 30 years...


Presumably that's what they are showing with two wires with different shape connectors


And/or the box labeled "Utility Power" is not intended to convey any specific notion of single circuits or multiple circuits nor of single substations or multiple substations. It just depicts power that in some way comes from a utility and no more.

In that sense, it is not incorrect about the configuration of utility power because it doesn't say anything about that subject.


I read the graphic differently and it reads ok to me.

The two different power sources are different shapes.

To someone who’s setup things in datacentres, it seems pretty reasonable to see that could be 2 different circuits.

Of course it should be 2 separate plugs with the 2 different shapes in them.

As for batteries you can just get a UPS system that supports adding extra batteries to it.


[flagged]


Fun fact, Germans are also the biggest consumers of fetish porn whose production they outsource to poorer Eastern European countries.


Western Germany has produced porn even during the socialist years in Hungary.


Huh?

How did we subsidized east European erotic call centers?


By calling them


That's just wrong use of words.

And weird accusations like we Germans did this on purpose like mafia


Are you saying Germans subsidized erotic call centers by accident?


No. I'm saying that German did not subsidize those at all.

There is not a source I read about Germany as a country did this.

I understand that German people were using those lines and made it therefore profitable but that's not what the word subsidize implies.


Look. I understand that some Germans were not at all involved in subsidizing erotic call centers, and those innocent Germans should be left in peace. But it's not at all clear how many Germans have been subsidizing erotic call centers, nor how much the German government had a role.


Without any sources this is speculation.

And regarding single Germans: they probably also not subsidized it.

They were either customers or investors


> Without any sources this is speculation.

I completely agree that we need more data.

The German government has never issued a firm and unequivocal denial that they subsidize erotic call centers, neither now nor at any time in the past.

It is interesting that more moderate Germans do not condemn the more radical erotic call center supporters amongst their countrymen.

Their silence on this matter speaks volumes.


I have anecdotal evidence to offer.


We all do, German and non-German alike.


I’m confused how we ended up here, but happy we did. Does anyone have more info on the history of German subsidization of interactive auditory erotica?


I really think it's time we had answers from the German government on this question.


haha




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: