No. JRuby and Rubinius both have benchmark suites, but I believe they don't go as far as kernels from real gems, and neither of them track benchmarks in any kind of continuous integration system, which is why I developed Bench 9000 as part of my PhD.
But if they were to benchmark and see that things like that pack method were slow, I think it is unlikely they would be able to implement the algorithms needed to improve on this kind of code, given their current implementation techniques.
Rubinius is essentially a template compiler, emitting a chunk of LLVM for each byte code. There isn't any sophisticated optimisation before it goes into LLVM, so nothing to for example partially evaluate a sort routine or remove allocations. The LLVM that comes out is far too complex for LLVM's optimisations to work for them.
JRuby relies on the JVM to do the sophisticated optimisations, and C2 (the server compiler) just doesn't have the optimisations or inlining scope needed to simplify code like the pack example. JRuby are massively improving on this with their IR, but they are going to have reimplement some very complex optimisations themselves to make this work on methods like pack.
>>But if they were to benchmark and see that things like that pack method were slow, I think it is unlikely they would be able to implement the algorithms needed to improve on this kind of code, given their current implementation techniques.<<
That may be.
I think it unlikely they would be unable to improve on this kind of code without performance tests for this kind of code.
I'm not sure what you're getting at any more. These aren't benchmarks we've pulled out of nowhere. It's existing Ruby code that people are running to make money right now. Any other implementation could have tried to improve on the performance by running it just as we have.
But if they were to benchmark and see that things like that pack method were slow, I think it is unlikely they would be able to implement the algorithms needed to improve on this kind of code, given their current implementation techniques.
Rubinius is essentially a template compiler, emitting a chunk of LLVM for each byte code. There isn't any sophisticated optimisation before it goes into LLVM, so nothing to for example partially evaluate a sort routine or remove allocations. The LLVM that comes out is far too complex for LLVM's optimisations to work for them.
JRuby relies on the JVM to do the sophisticated optimisations, and C2 (the server compiler) just doesn't have the optimisations or inlining scope needed to simplify code like the pack example. JRuby are massively improving on this with their IR, but they are going to have reimplement some very complex optimisations themselves to make this work on methods like pack.