I haven’t seen a Blame! reference in the wild before! The concept of runaway AI that never stops building more infrastructure seems like one of those dystopian scenarios that is at least semi-plausible, and the idea that humans effectively lock themselves out of control by being too clever (net terminal gene) is just the cherry on top.
Cars have a lot of trim/seals/insulation/carpeting to reduce road noise and be aesthetically pleasing, and military aircraft cockpits don't care about either of those things, and are largely just a metal tube with racks on which all of the avionics and other cockpit equipment are mounted, with holes in a handful of places where wiring harnesses enter/exit the cockpit. All of that equipment is regularly worked on, removed/replaced, and so it is necessary that it be (relatively) easy to access and remove.
The equivalent would be like if you had to pull all of the instruments, electronics, and seats out of your car every 1000 miles, clean them up, replace faulty bits, and then put it all back again. All of the fancy trim, carpeting, etc., just makes that job harder, so you would probably want a car that doesn't have any of that, and is designed to make doing that kind of work easier, better still if you can avoid having to remove everything, and only have to remove the bits individually that need to be maintained. The down side of course, is that without all of the fancy trim and stuff, there would be gaps where things could fall and be hard to reach, and holes where wiring travels to the engine compartment/trunk/etc. Of course, FOD presents way less of a danger in a car than it does an aircraft, so you might not care if you drop something there, but aside from that, I think the analogy holds up.
I worked on F-16 avionics (in aircraft maintenance, on the flightline), there are a lot of little nooks/holes/slots/gaps where small bits of FOD can fall and be incredibly difficult to extract, and the fear of that FOD causing a jam, flying around the cockpit, or getting wedged and causing unexpected wear on wiring harnesses and then shorting out (or worse, arcing) during flight was a _big_ deal. FOD in the cockpit was basically the worst thing that could happen during routine maintenance, because if you couldn't see it, and either couldn't get to it with a magnet (or the FOD wasn't metal), it might require pulling _a lot_ of stuff out of the cockpit before you could reach into the area where it fell. The worst case that could easily happen was having to have Egress come out and pull the ejection seat so you could get under it.
I always figured that all of those little gaps/etc. were due to a couple factors:
1.) the aircraft are constantly being upgraded/modified, so even if you designed the aircraft to be gap-free initially, there will inevitably be changes that introduce them. The cockpit itself is basically a frame with racks that hold all of the avionics, seat, etc.
2.) in conjunction with the above, ease of maintenance was somewhat important, so they tried to leave at least a little room to maneuver in the cockpit where possible (though there were plenty of places which were a nightmare to work regardless), but that comes at the cost of introducing areas where things can fall.
3.) some components have to be regularly removed and worked on outside the aircraft, or must be free of obstruction during flight, e.g. the ejection seat. So you end up with plenty of gaps where things can fall.
> If we pretend the Supreme Court is partisan, which it is not but let's entertain that thought here..
I mean, that’s laughable, I don’t know how anyone can call the current Court non-partisan with a straight face. That doesn’t mean every decision is necessarily partisan, but multiple decisions over the past few years have been blatantly partisan, with Thomas and Alito especially making very little effort, if any, to hide it.
I used to feel that the Supreme Court was the one of the few non-partisan institutions in our government, but the way this Court has run roughshod over stare decisis to effect change in the interests of its majority was more than enough to disabuse me of that notion.
Everyone loves when decisions are going their way, consequences be damned, but the way those decisions get made matters, and nobody should be happy with the way the current court is choosing to act. It has set a really bad precedent for future courts, and has already done immense damage to its reputation in the eyes of many.
What are you talking about? I’ve been on the same dosage for over 12 years, and it is just as effective for me today as it was on day 1. Still waiting for those consequences I guess..
I mean, the BEAM doesn't have global GC pauses either, as each process has its own heap - but I would expect Pony can take things a step further as a result of its strong type system, which IIRC is why it can support zero-copy messaging.
This is true. Erlang's heap per PID. Azul's C4 and other JVM GC move in the no world stopping direction but they're still at the mercy of the model of the JVM.
If one can avoid GCs altogether a-la precise (de)allocations like Rust's non-reference-counted entities, this is cool but often requires unnatural contortionism. RC is still necessary in certain cases.
I really strongly doubt that GC is a bottleneck for Erlang programs on either the BEAM or the JVM - the sophistication of the scheduler, and the way various language primitives interact with it, is where the BEAM is almost certainly gaining an edge over the JVM. That said, I'm sure there are a subset of programs that _would_ be faster on the JVM, just depends on what metrics are being compared.
> the sophistication of the scheduler, and the way various language primitives interact with it
That was brought over to the JDK six months ago. The JDK can now spawn millions of Erlang-like processes ("virtual threads") per second.
Erlang is a great inspiration and it does incredibly well with the development resources available to it, but it's hard to compete with the level of engineering investment in the JDK and its state-of-the-art GCs, optimising JIT compilers, and low-overhead in-production tracing and profiling.
The BEAM is so much more than just a green thread runtime - and it is not straightforward at all to just "bring it over" to another virtual machine, the entire BEAM VM is designed around the scheduler, and core features such as signals (used most notably for links/monitors, but also for a number of other system features) and messaging, are deeply integrated with the scheduler; as are system tasks, execution of NIFs (natively-implemented functions), and more. Furthermore, the schedulers are adaptive to the amount of work in the system, and the behavior of the other schedulers, i.e. they aren't just simple work-stealing queues for green threads. Over 20 years of engineering effort have been invested in this system, and it is highly optimized for the types of workloads that Erlang is used for.
In my opinion, trying to replicate ERTS on top of another virtual machine either requires making that virtual machine more like the BEAM, or will end up always playing catch up with the BEAM itself in some aspect. That's just my two cents though.
I am well familiar with BEAM, and my fascination with it ~20 years ago was one of the things that inspired me to become a VM engineer. Bringing that over to the JDK was, indeed, a large effort, but we are a large, well-funded team.
And, as a JDK engineer with Erlang experience, I'm just sayin' that as of the past six months, the JDK has all the primitive building blocks required for a very efficient Erlang runtime. Even the thread scheduler is not built into the JVM but written in Java, and will soon be made completely pluggable. Any scheduling algorithm that Erlang wishes to employ can be provided as a library on top of the JVM. If I've missed anything, we'll consider adding it.
It's been a dream of mine to port BEAM to the JDK to give Erlang much better performance and greater visibility. Unfortunately, this work can't be funded as there's insufficient demand for Erlang. If the Erlang community does it, however, it could increase its evolution velocity, improve performance, and give it higher visibility.
BEAM doesn't support mutation, the JVM does. that's a big plus in the BEAMs corner.
Also when two BEAM VM speak to each other its as if they are just one machine. no serialization de serialization of data, no protobufs, etc. It's all just primitives being sent between processes regardless of their location.
> BEAM doesn't support mutation, the JVM does. that's a big plus in the BEAMs corner.
The JVM supports languages with mutation or without. That x86 supports mutation doesn't hinder, say, Haskell from being efficiently compiled to it.
> Also when two BEAM VM speak to each other its as if they are just one machine. no serialization de serialization of data, no protobufs, etc. It's all just primitives being sent between processes regardless of their location.
There is no need for that protocol to be built into the VM . The JVM is a lower-level VM than BEAM (parts of the runtime, like the thread scheduler and even one of the JITs are written in Java).
I have the utmost respect for Erlang's design, and have borrowed inspiration from it. It is truly a towering achievement. But BEAM simply doesn't have anywhere near the level of investment required to be a leading platform. I wish there was more demand for it, but there isn't.
It's important to highlight some key differences between the JVM and the Erlang VM (BEAM) when it comes to accommodating language semantics, fault tolerance, and process isolation.
The JVM is designed as a general-purpose VM, aiming to accommodate a wide variety of languages with different semantics. This broad scope can sometimes lead to compromises when it comes to optimizing for specific language features or use cases. On the other hand, the Erlang VM is tailored to the semantics of Erlang and fault-tolerant systems, which allows it to be highly optimized for these specific requirements. This focused approach enables the BEAM to excel in areas such as concurrency, fault tolerance, and process isolation.
As mentioned earlier, the BEAM offers strong fault tolerance and process isolation. When a specific library or process encounters an issue and panics, it doesn't bring the entire VM down. Instead, only the affected process is terminated, allowing the rest of the system to continue functioning as expected. This level of isolation is achieved through BEAM's lightweight processes, which have their own heap and stack, and communicate with each other only through message passing.
Some features that the Erlang VM has, which are specifically tailored to fault-tolerant systems and not found in the JVM, include:
a) Per-process garbage collection: BEAM uses a per-process garbage collection approach, which enables it to efficiently handle millions of lightweight processes without the need for stop-the-world GC pauses.
b) Preemptive scheduling: BEAM's preemptive scheduler ensures that no single process can monopolize the CPU, and that all processes receive a fair share of processing time. This is a crucial feature for building highly concurrent and fault-tolerant systems. (The BEAM is ostensible an OS and can even be run bare metal!)
c) Supervision trees: The Erlang/OTP platform provides a built-in supervision tree mechanism, which allows developers to define strategies for monitoring and restarting failed processes automatically. This contributes to the overall stability and reliability of the system.
The JVM has made significant advancements in performance and garbage collection, the BEAM has been specifically optimized for the unique requirements of Erlang and fault-tolerant systems. Its focused approach allows it to offer features and optimizations that are particularly suited for these use cases.
In response to:
> But BEAM simply doesn't have anywhere near the level of investment required to be a leading platform. I wish there was more demand for it, but there isn't.
While it's true that BEAM may not have received the same level of investment as the JVM, it's important to note that the level of investment doesn't necessarily determine a platform's effectiveness or suitability for specific use cases. The Erlang VM has been designed and optimized over 35 years for the unique requirements of Erlang and fault-tolerant systems, which has resulted in a set of features and optimizations that cater specifically to these needs, its unparalleled in this field, and As such, the BEAM's focused approach allows it to be a strong contender in its niche, particularly for building highly concurrent, fault-tolerant, and distributed systems. It's not necessarily about competing with the JVM across the board; instead, it's about excelling in specific areas where the Erlang VM's unique features and optimizations provide tangible benefits.
Moreover, the Erlang/OTP ecosystem has a dedicated and passionate community that continues to drive its development and innovation. This community's efforts have ensured that BEAM remains a powerful and relevant platform for building resilient systems, despite having comparatively less investment than other platforms like the JVM.
OpenJDK's new GC doesn't require STW pauses, the JDK offers preemptive scheduling for millions of threads as well as supervision trees -- although, to be fair, these are recent changes that have, at least in part, been inspired by Erlang. All that is on top of one of the world's most capable compilers. I have long admired Erlang and its passionate community as well as how much they've been able to accomplish (and, as a VM engineer, studied its design), but between passion and being an actual top-tier competitor there's a certain reality gap.
To be a competitor, to be fair, implies there's competition. ¯\_(ツ)_/¯.
There's no world, where I'd allow for some future developer to write some Java on the same VM as my Erlang/Elixir code and risk the entire VM crashing.
JVM and Erlang's VM cater to different needs and programming paradigms. While they might share some features or learn from each other's innovations, they are not direct competitors but rather address unique use cases and development requirements.
People who want Erlang are going to reach for Erlang/OTP.
People who want Scala/Akka/Java-monolith will leverage the JVM.
Replatforming, doesn't happen often.
No one is hopping from one platform for the other.
What do you mean by "the entire VM crashing"? The JVM is no more susceptible to crashes than BEAM.
BTW, my point is that can now be a very good platform for Erlang/OTP. Its performance will allow Erlang/Elixir programs to reduce their reliance on native code and to handle higher loads.
It's my understanding the state of the art in observing JVM-based applications is a combination of using thread dumps, gc logs, thread activity visualizations. Thread dumps give us a snapshot of the the name of the thread, its current running state (waiting, blocked, etc), and the stacktrace of the work its currently doing. GC logs give you a record of when and how much garbage was collected and Thread activity visualizations show you the timeline of thread moving between different running states.
The BEAM gives you the ability to see the bottlenecks in your system, via the REPL (in real time!)
It has world-class introspection built in that gives you the power to observe and manipulate your running application through a REPL.
The BEAM has hundreds of features like this, because the BEAM is more of an OS than and VM.
I get it, you're a JVM expert, but the BEAM is more than a check list of optimizations that on paper the JVM can do.
I strongly suggest, before the next time you comment on an BEAM VM vs.JVM debate, please consider watching https://www.youtube.com/watch?v=JvBT4XBdoUE, "The Soul of Erlang and Elixir • Sasa Juric • GOTO 2019"
and reading https://github.com/happi/theBeamBook " an attempt to document the internals of the Erlang runtime system and the Erlang virtual machine known as the BEAM."
You can do all that with the JDK, too, and at higher performance.
As an Erlang fan for many, many years, I am quite familiar with BEAM's design (and why I think Erlang would gain a lot by running on the JVM (or, more precisely, from implementing BEAM and OTP on top of the JVM). Aside from implementation details -- the JDK has simply had a lot more work put into it -- the main difference between the two is that BEAM is higher level, and the runtime itself does more things out of the box, while the JVM is lower level and you can do more in libraries (with no loss of performance, because Java is compiled to very efficient machine code).
Not only would Erlang and Elixir gain better performance and be able to reduce application's reliance on NIFs, but maintaining and evolving them would require less effort.
There is now (as of JDK 19, released six months ago) no capability offered by BEAM -- including runtime manipulation of code through a REPL -- that couldn't be implemented at least as efficiently as a library on top of the JDK. You can see the API here: https://docs.oracle.com/en/java/javase/20/docs/specs/jvmti.h...
You’re moving the goal post. The JVM does provide real time observability into your application.
Personally I think in a terminal with just text is inferior to a proper gui tool. If you really want this jcmd can convert jfr data into different formats that you then can script against.
It's the actor model all the way down... that's the abstraction.
Simple systems, are easy to manage, complex systems are harder to manage, hence why you need a full blown GUIS to manage the JVM, because it does everything, and tries to satisfy every use case, so it has to handle every use case.
The author of the article you linked is about a decade out of date. "The state of the art in observing your JVM-based applications is a combination of using thread dumps, gc logs, thread activity visualizations." -- that has been untrue for quite some years now. The basic observability offered by the JDK is now the Flight Recorder, a low-overhead in-production tracing and profiling engine that BEAM is many years away from being able to achieve.
The JVM supports all the same runtime observation and manipulation as BEAM. If you want to use them via a shell -- you may. If you want to use the actor model "all the way down" you can do it. The JVM is simply a virtual machine at a lower level than BEAM, but it does provide all the primitives the Erlang model needs.
The reason I'm telling you this is that I think that targeting the JVM would allow Erlang/Elixir to evolve faster -- enjoying the low-level work done by the large OpenJDK team -- as well as reach a wider audience. At the same time, the wider Java platform community will be exposed to all the great ideas that have come out of Erlang. All the necessary foundational work is now, finally (as of JDK 19), in place.
It is a goal post move. You went from "Beam provides real time observability" to "real time observability is only in a terminal with a repl" when provided with facts that the JVM also provides real time observability.
> a "Proper GUI tool" is not.
I don't know what this means.
> hence why you need a full blown GUIS to manage the JVM,
Did you read the post you linked? The author required a GUI written by someone else.
I don't know what point you're trying to make anymore, you're just spitting out random sentences, that don't have to do with anything I was replying to (real time obserability).
You can write a library/tool that does exactly that, and the JVM offers an API precisely for that purpose. The reason such a tool isn't offered out of the box is that it trades off performance. For example, the JIT can optimise a variable or even an entire object away. When you want to inspect them at runtime, this would trigger a "deoptimization", where the representation would go back to the original bytecode from the optimised machine code. Observability that doesn't trigger deoptimization is, therefore, encouraged, but everything is available.
I don't know what that means. The JVM itself is at least as reliable as BEAM, and probably more so -- it's subject to both more development testing and field testing -- but what needs high reliability is applications. The JVM runs much more critical code than BEAM does, and on a far wider scale and more stressful workloads. The VM itself is the cause of neither Java's reliability nor Erlang's. It's the programming model that matters. The JVM can now support the Erlang model at least as well -- and, I claim, significantly better -- than BEAM ever could with its limited resources. BEAM's Achilles heel has always been performance. That has harmed Erlang application's reliability, as they need to depend on unsafe native code -- that can crash the entire process -- much more than JVM applications.
> The VM itself is the cause of neither Java's reliability nor Erlang's. It's the programming model that matters. The JVM can now support the Erlang model
That's a common misconception. There is a huge practical difference between "being able to write share-nothing" and "having whole ecosystem built around it". While you're referring to Erlang running unsafe native code, you're completely ignoring the Java running unsafe Java code. To ensure guarantees like "share nothing" or "restartable processes" Erlang developer just needs to write only Erlang and Java developer needs to carefully pick every dependency and almost always rewrite some of them.
That's the exact reason why Akka didn't replace OTP: you're most likely using the dependency which is not written with Akka, and it might share mutable memory, use global mutable state etc. So, even with possibility of writing Erlang-like software, you most certainly will end up with Frankenstein where some parts are fault tolerant, immutable, functional and reasonable, and some parts are oldschool java 8 bullshit.
Do you know what is the definition of DSL? It is when you write your code in one language and get exception in another. And that's exactly why almost every JVM language (except Java) is a DSL.
You just can't have guarantees Erlang has without rewriting everything with your Akka of the day.
I think you misunderstood my point. I wasn't trying to argue for Java over Erlang. The people writing Java will continue writing Java, and people writing Erlang will continue doing that.
I was merely pointing out that at this point, the JVM offers a great platform for Erlang itself, which could both increase its visibility and give it a technological boost. You could have everything be Erlang, top to bottom, with far better performance than BEAM could ever achieve, less work, and better interop with where a lot of people actually are.
There could be some disadvantages but overall I think it would be a great opportunity, and one that Erlang could really use.
> And that's exactly why almost every JVM language (except Java) is a DSL.
Other JVM languages choose to have "exceptions in Java" because it's so much less work for them to use existing libraries and they consider it a small enough price to pay. But even if you want virtually all of it to be Erlang, it would still be less work than BEAM.
You could even control exceptions and stack traces, as that's programmable:
Great, so you guys can spend the next 30 years writing OTP to work seamlessly and flawlessly on top of the new JVM with the same (memory, security, reliability, fault tolerance, etc) guarantees OTP has on the BEAM VM today :)
Erlang only uses reference counting for binaries larger than 64 bytes, everything else is allocated on the process heap (or in heap fragments) and copied. Just that is enough to have a beneficial effect though, since large binaries are relatively common in practice, and are frequently passed around from process-to-process.
And if you have any other global state you want to pass around, you can pull off a clever trick by passing it around as a binary and then unpacking it as needed within caller processes.
AFAIK this trick is why BEAM files use an IFF-derived format (easy to parse individual chunks out at runtime), and why erlang:module_info/{1,2} are the way they are: working with module metadata literally just means asking the code-server process for the (shared refcounted) module binary, and then parsing it yourself.
I think that's largely an implementation detail, so not a requirement as far as I know. That said, delimited continuations certainly make them easier to implement, as I understand it.
I took a look at the paper and it too shows that the effects are just delimited continuations with named, algebraic-typed prompts. It even goes into the difference between shift0 and shift, but not by that name. (It claims that Koka and others are using shift, which isn't quite what I remember but okay.)
I guess it's a good marketing move because "multi-prompt delimited continuations" is scary. It also makes the types sane; I once worked out that expressions with delimited continuations of the reset-shift0 variety are typed by binary trees of ambient types, with a subtyping relation s.t. leaf nodes can be expanded.
If you are handwriting the function in assembly, you'll know what registers hold the function parameters, what types of values they are supposed to be, and with care, you can produce debug information and CFI directives to allow for stack unwinding, it's just annoying to do - but that's just the tradeoff you make for the performance improvement I suppose.
