One of the things I’ve been hoping for every time a new EC2 instance comes out is for them to unpin the memory:core ratio a bit. I don’t expect they have enough r# and c# users to completely balance things out so what they’re really doing is selling people more CPUs to get the memory they need.
It would be nice if it were creeping up generation to generation. But if this keeps up I fear the opposite.
For many ephemeral workloads, sure, but that comes at the expense of generally worse and less consistent CPU performance.
There are plenty of workloads where I’d love to double the memory and halve the cores compared to what the memory-optimised R instances offer, or where I could further double the cores and halve the RAM from what the compute-optimised C instances can do.
“Serverless” options can provide that to an extent, but it’s no free lunch, especially in situations where performance is a large consideration. I’ve found some use cases where it was better to avoid AWS entirely and opt for dedicated options elsewhere. AWS is remarkably uncompetitive in some use cases.
kv stores also exist because for many generations of tooling it was faster to manage read-mostly data off-heap instead of on, and that becomes more true the more processes you run doing jobs that touch the same data.
Yeah those are pretty spendy. I know one comes with extra guaranteed bandwidth which is kind of handy if you’re sharing a small number of cache nodes among a lot of servers. But we were doing okay running r6 for cache, though my coworker who knew the ritual for migrating them did eventually get a little boost out of switching us to r7’s. The latency wasn’t great and I don’t think faster network cards would have helped that. There was already plenty of incentive for us to do per-request promise caching to avoid pulling the same keys multiple times in a request but that was necessary because the business model forced the architecture to tolerate nondeterminism. The cost per request was what eventually killed them (the economy dipped and customers ran to cheaper vendors), but I’ve never seen a company survive being stupid for as long as this place did.
You should just get used to it because the memory per core is going down inexorably forever until someone makes a physics breakthrough. We know how to print cores and the core count is going to keep going up.
You can build eDRAM using logic processes. It's not usually done since ordinary DRAM ends up being cheaper, but if the usual DRAM processes are bottlenecked (and SRAM cell scaling is also hitting roadblocks of its own) that makes eDRAM a lot more viable, at least for specialty uses.
It would be nice if it were creeping up generation to generation. But if this keeps up I fear the opposite.