> The thing I don't get about Erlang and BEAM is the idea that having lots of little processes means that your program will scale brilliantly to run in parallel.
The point is scaling. Think in terms of request rate. If you know you can have millions of processes per machine and they run well in parallel, then you can handle requests with processes and stop worrying.
Erlang generally encourages shared-nothing architectures. Of course, in some cases you want regions of shared memory or some other concurrent global resource, hence ETS. I see nothing wrong with ETS, it's well optimized for the Erlang term format in particular and gives you serializable updates.
Scalability and large actor counts aren't the definitive features of Erlang, though. It's supervision trees, the distribution protocol, the OTP framework, the primacy of tuples and lists as your main and highly flexible data types, a great pattern matching engine for binary formats and regular Erlang terms alike, module-level hotswapping, a crash-only programming model, the ability to have external programs benefit from Erlang semantics via external nodes and ports, so on and so forth.
Yes, not all of this is thanks to the VM in of itself. A lot of it is runtime and language features.
But it's already there in a cohesive whole. There is absolutely no reason to switch to the JVM when the EVM is a beast of its own.
> Erlang generally encourages shared-nothing architectures.
This is the language feature that `pron` keeps mentioning. Nothing about the VM is especially better for this than the JVM for instance.
> I see nothing wrong with ETS, it's well optimized for the Erlang term format in particular and gives you serializable updates.
There isn't anything wrong with it (as a complete neophyte to ETS and the EVM generally), the question is how much better it could be if it was on one of the several first rate JVMs that get so much more resources poured into them. Sharing data concurrently is precisely what the JVM is good at (especially at very large data set size). So in the cases where you need to use something like ETS, there is a lot of potential for improvement on a JVM vs EVM.
> But it's already there in a cohesive whole. There is absolutely no reason to switch to the JVM when the EVM is a beast of its own.
I don't want to speak for `pron` but I suspect what he is getting at is, the combination of the Erlang full story on the JVM would be a phenomenal bit of tech and it would be much easier (and more likely) for the Erlang bits to get ported to the JVM than it would be to bring the EVM up to the standard of any of the best JVMs.
It would be a phenomenal bit of tech anywhere. We had to go with Erlang for one piece of our product at Plum precisely because there is no analog story in Haskell to the Erlang full story. I would have loved to be able to build that specific piece in Haskell but it made little sense when considering what Erlang/OTP provides.
It's nice to say that the JVM has more resources and is better at XYZ while the BEAM VM is only better at ABC, therefore the Erlang full story should be on the JVM to reap the benefits of both; however, I think that would be unhealthy for Erlang. Different VMs present specialized focuses and I think the areas that BEAM is lacking in can be tackled and brought up to parity instead homogenizing the VM-field and adding to the kitchen-sink that the JVM already is.
> Probably. But why not spend that effort on the language and libraries?
Because the best Erlang has to offer isn't really in the Language. Some in the Libraries. Most in its VM. I would pick Haskell or ATS over Erlang - unless I need those few unique features that Erlang/OTP really got right.
> That makes as much sense as saying your language shouldn't run on the kitchen-sink Linux so as not to homogenized the OS field.
No, your analogy is moving the goal post. No one ever said someone couldn't implement Erlang the language for the JVM. As many people have pointed out to you, it isn't the language that makes Erlang. It's BEAM + OTP. We're not talking about moving the language around, we're talking about gutting the VM, my statement still holds: Exalting the JVM to be the one-true-VM for Erlang (therefore also implying any language you do not understand well that needs a VM) is a very bad idea and pretty silly.
Diversity is good. BEAM's VM is good. The JVM is good. Even the CLR is pretty amazing (F# beats the pants off of Scala). There's no reason at all to think that Erlang would be better off on the JVM; however, borrowing successful ideas from other awesome and successful technologies? I think that's a swell path to walk. Beware of the kitchen sink, though, is my only warning.
Also, I don't always think OS' are the best place to run your application. There are many arguments for using something like Erlang on Xen or HaLVM if the design requirements can justify it but arguing about your other critical statement should be a different thread.
Good points. Monoculture hurts us. BEAM people could be putting more effort into copying improvements in competition and academia. That would get BEAM performance way up there. Right now, they seem to focus more on other things with BEAM performing good enough for its users. With the diversity benefit on the side.
You would be surprised, actually. I think there's quite a bit of work going into BEAMJIT, an LLVM based JIT for Erlang that far exceeds HiPE. I won't argue with you though that the VM could use improvements gleaned from the last ten-fifteen years of implementation and research.
To me, it really just seems like a difference in labor and time. BEAM doesn't have nearly as much investment of brainpower going into it. It's also been mainstream for much less time. There's less corporate R&D working on it. So, we'll see things progress more slowly and it be behind in various ways until some of this changes.
True for most stuff out there that's not the No 1 or No 2 choice for mainstream developers. Yet, results such as HipHop and PyPy show vast improvement can be made when even one company puts a lot of effort into something. A combo of academics applying to Erlang/BEAM the best in FP compilation and best in VM architecture might bring similarly dramatic improvements. A good precedent is how I saw the Racket Scheme team knock out significant weaknesses in their compiler practically as they were posted on forums. Not saying it always happens but that sort of thing in Erlang space would be interesting.
> There is absolutely no reason to switch to the JVM when the EVM is a beast of its own.
I think there is, if you want to concentrate your limited resources on the language and its phenomenal libraries while letting an enormous team working on the world’s second-largest open-source project take care of the VM for you, while at the same time giving you better performance and a wider reach. There are many more organizations that would adopt Erlang if it were on the JVM.
Completely ignoring the massive resources it would take to make the switch in the first place.
Erjang doesn't cut it. It's an incomplete research project that works on the basis of bytecode translation. Further, the disadvantages with regards to global GC are clearly listed. You say that it'll only keep improving, but that's essentially taking a leap of faith that the JVM developers will eventually get to parity with a feature you already have.
"Limited resources" is a red herring and FUD, plain and simple. Nor is "wider reach" guaranteed in the slightest. Wider reach is not intrinsically a good thing, either. Organizations for whom Erlang is out of reach simply because it doesn't use the JVM are absolutely petty and there is no loss from them not using it, IMO.
> Completely ignoring the massive resources it would take to make the switch in the first place.
If I'm suggesting it, I obviously believe that the cost/benefit is worthwhile. I don think the effort required is massive.
> You say that it'll only keep improving
I say that it has already improved enough.
> Organizations for whom Erlang is out of reach simply because it doesn't use the JVM are absolutely petty
Not petty, but rational. Those organizations already have millions of lines of code, and lots of knowledge and experience on the JVM, and the reasons for choosing Erlang aren't compelling enough given the adoption cost. But if you lower those costs...
All I'm suggesting is simply lowering the adoption costs of an upcomer, rather niche tech with a small ecosystem (which would also, I'm convinced, considerably improve the tech).
> [...] but you can have those millions of processes on the JVM, too [...]
In theory, yes. In practice, it's very difficult. JVM thread corresponds to
a thread of host OS. Spawning and keeping those is very slow and expensive
compared to Erlang's processes. You could have a pool of pre-spawned workers,
but suddenly you can't spawn a worker for each connection and hope it will all
work; you need to manage the pool. You also could try to implement green
threads in JVM, as they are in Erlang, but you would need an entirely new
compiler to insert yield points at appropriate places, or else you would get
exactly the same problems with green threads as everywhere else.
Under JVM you just don't spawn a thread for each and every activity, because
you would choke your system. The whole point of developing Erlang was to allow
exactly this programming style in a manner safe against processing
congestion.
> you need to manage the pool. You also could try to implement green threads in JVM, as they are in Erlang, but you would need an entirely new compiler to insert yield points at appropriate places.
I am talking about a new compiler. An Erlang compiler. The JVM is a virtual machine. Erlang is a language. We've already proven you can run Erlang on the JVM quite well, and that was before many pertinent improvements to the JVM and its ecosystem.
JVM still lacks some serious functions, namely, links and monitors. I'm not
that sure you can emulate those on JVM reliably without implementing them as
fundamental operations.
And then, there's still problem of interoperability. Even when you write your
code in Erlang@JVM, it still needs to talk to Java code, which doesn't have
yield points.
> JVM still lacks some serious functions, namely, links and monitors. I'm not that sure you can emulate those on JVM reliably without implementing them as fundamental operations.
You don't emulate them; you implement them -- just as BEAM does. Having them baked into the runtime serves no purpose. The JVM operates at a much lower level than BEAM -- just as BEAM is implemented in C, it could be implemented in Java, except that the really hard parts (JIT and GC) are already taken care of. Think of Java as C + JIT + GC.
> it still needs to talk to Java code, which doesn't have yield points.
That's not a problem. First, Erlang code talks to C code, which doesn't have yield points, either. Second, the JVM doesn't need to rely on yield points as much as BEAM does, because it is much more kernel-thread-friendly than BEAM.
Akka actors are multiplexed on real JVM threads. So they implement a cooperative model of threading (i.e. you'd better not block for long inside the body of an actor)
I'm guessing you're the Quasar guy or am I off? Either way, I think you have benchmarked the lightweight thread approaches on JVM's. How much simultaneous concurrency can the JVM methods manage right now for say serving web requests? And how much does Erlang's best do on same machine?
I think that's an interesting and useful comparison point to start with to test your claim. This is also something I figured Java side would greatly improve on.
The point is scaling. Think in terms of request rate. If you know you can have millions of processes per machine and they run well in parallel, then you can handle requests with processes and stop worrying.