The situation is the opposite of what you seem to think. It's far easier and cheaper to scale up than out than it has ever been. Just 5-6 years ago it would have been nearly unthinkable to have a half a TB of memory, 16 CPU cores, and a TB of SSD for a few thousand dollars.

Most people's traffic/data hasn't grown nearly as fast as Moore's law. Scaling up makes more sense than ever. It may not be hip but it's the right choice for 99% of cases.

Just be ready for the day when you max out your IO bandwidth, or some other hard-to-forsee bottleneck.

There is always a bottleneck. Moving it from "disk I/O" to "cache coherency network traffic" doesn't magically make that go away.

Bottlenecks are not magically confined to only affect scale-up designs. And if you look at the design of scale-out systems, they usually surrender some desirable quality such as easy joins, isolation, atomicity etc.

It depends on how fast you're growing, since technology keeps chugging along too.

Case in point, here's a 16TB SSD with 560 MB/s peformance: http://www.engadget.com/2012/01/09/ocz-goes-ssd-crazy-at-ces...

You might be able to keep cramming in more RAM until memristor storage makes RAM obsolete.

I know it's a cliche, but memristors are going to cause an earthquake in the algo-economics landscape.

I have a hard time seeing whether memristors are really going to be the future of all storage or whether it's just a fad. Can somebody who knows something about the topic explain this?

Here's a useful, but lengthy, talk on the subject: http://www.youtube.com/watch?v=bKGhvKyjgLY

Memristors look extremely likely to be able to provide non-volatile storage with latency, access speeds, and density higher than existing DRAM and flash. They also look like they can be used to make FPGA-like devices which can approach the speed, power efficiency, and logic density of custom designed ASICs but with extremely fast reconfiguration speed. And those seem to be the less fascinating aspects of memristor technology.

The thing about memristors is that they could potentially have transistor-like speed at the microlevel, but they can hold data at much, much higher density without constant input of power.

Potentially memristors could lead to smartphones and tablets with the power of a server rack of today crammed into them, but also with improved battery life.

And that's one of the least revolutionary implications of memristor technology.

unforeseen problems also happen with horizontal scaling

They've proudly blogged about it a few times http://blog.stackoverflow.com/category/server/

I think they've successfully shown how far you can take it. Nevertheless, I agree with you that scaling this horizontally makes a lot of sense.

They ARE scaling horizontally. Remember where they mentioned that they are putting StackOverflow, one of several sites that they run, on its own box? That is horizontal scaling. They now have two shards, one with SO and one with all the other sites.

No matter how you slice it, sharding introduces substantial complication. You need to modify your application to shard or to use an auto-sharding store, and then maintain that overhead forever.

The thing is, they don't need to do that. Moore's Law, here represented as SSDs and RAM, is keeping ahead of their traffic growth.