Is that really a viable solution for a timeframe of 6+ hours?

Sure, if you know how much you allocate per minute (and don’t exceed your budget) you just buy enough RAM and it’s fine.

This will decrease performance because of reduced locality. Maybe increased jitter because of TLB misses.

Compared to what, running a garbage collector?

No, compared to not doing so many allocations that freeing them is time consuming or expensive. Having allocations slow a program down means that there are way too many, probably due to being too granular and being in a hot loop. On top of that it means everything is a pointer and that lack of locality will slow things down even further. The difference between allocating many millions of objects and chasing their pointers and doing a single allocation of a vector and running through that can easily be 100x faster.

Probably? Locality becomes fairly important at scale. That’s why there’s a strong preference for array-based data structures in high-performance code.

If I was them I’d be using OCaml to build up functional “kernels” which could be run in a way that requires zero allocation. Then you dispatch requests to these kernels and let the fast modern generational GC clean up the minor cost of dispatching: most of the work happens in the zero-allocation kernels.

(this comment was off topic, sorry)

Is this relevant to OCaml?

ha ha oops I got confused

I think it is, but to be clear I think (from my very limited experience, just a couple of years before leaving finance, and the people with more experience that I've talked with) that c++ is still a lot more common than any GC language (typically java, since OCaml is even rarer). So it is possible, and some firms seem to take that approach, but I'm not sure exactly how besides turning off GC or very specific GC tuning.

Here is a JVM project I saw a few years back, I'm not sure how successful the creators are but they seem to use it in actual production. It's super rare to get even a glimpse at HFT infra from the outside so it's still useful.

https://github.com/OpenHFT

Yes. It is a very common design pattern within banks for systems that only need to run during market hours.

I talk about doing this in an audio context and get met with raised eyebrows, I'd love some references on others doing it, if anyone can share!

You can just add more RAM until it is viable.

Are you aware of how many allocations the average program executes in the span of a couple of minutes? Where do you propose all of that memory lives in a way that doesn’t prevent the application from running?