Or the case designed for mounting comes with extendable legs to support the weight. Or the monitor is designed on some kind of lever system that can be set to balance. Lots of options.

I kind of like your first idea. The motherboard would be vertically mounted, and then have 90 degree PCIe risers that are structurally reinforced somehow, so that the heavy GPU can also be mounted vertically. You could have 2 of them, one for the GPU and one for additional cards. The whole thing would be mounted via the VESA screw holes on the monitor.

Would probably mostly work for people with quite large monitors though. 40+ inch 4k monitors rock!

... or we just stick with tower cases and some kind of structural reinforcements for the heavy GPU.