RDP is aimed at a different use case than VNC. Proxmox and other virtualization managers (e.g. VMWare, Nutanix) use VNC because you get a stream directly from the hypervisor (e.g. KVM, ESX) which is very useful for low-level debugging. The VNC protocol also has very low overhead (you don't really want h264 encoding CPU overhead on your VM host). VNC is not really intended for remote desktop use cases, which require higher fidelity/frame rate, etc.
So -
* VNC: Low overhead / Low fidelity
* RDP (and other remote desktop protocols, e.g. Frame Remoting Protocol, Horizon Blast, Citrix ICA/HDX): Higher overhead / High fidelity
Low CPU overhead. VNC streams screen grabs with minimal (if any) compression, which results in lower CPU overhead, high bandwidth consumption and low frame rate. This is okay for the use-case of low-level VM debugging that it's used for in context of virtualization management systems, not so great for desktop remoting.
While RDP may run okay on 56k with low color mode for some use cases (e.g. simple Windows admin), it requires significantly more bandwidth and compute overhead (either CPU or GPU) for other more advanced use-cases (e.g. video editing, CAD etc.)
That might practically be where VNC finds usage today, but when it was introduced in the 90s, remote desktops were the intended use case.
"In the virtual network computing
(VNC) system, server machines supply not only applications and data
but also an entire desktop environment that can be accessed from any
Internet-connected machine using a simple software NC." -- https://www.cl.cam.ac.uk/research/dtg/attarchive/pub/docs/at... (1998)
Given the CPU load I've witnessed on VNC servers, I don't think "low overhead" is right these days.
VNC was designed for remote desktop use. All the other streaming features came along later. I don't see why RDP would make for a worse choice here, other than that Windows VM integration would make for an better solution.
RDP used to be far inferior because it was proprietary Microsoft stuff with buggy open source clients and undocumented servers that kept changing stuff around. These days, open source RDP server software is actually quite solid. I don't know if Gnome/KDE leverage the partial update mechanism that makes RDP so useful on Windows (doesn't seem to seeing the performance I'm getting out of VMs), but I find RDP to be a lot more useful for interactive desktop streams than VNC.
> I don't know if Gnome/KDE leverage the partial update mechanism
I guess that would be something for the wayland compositor to manage. Maybe a wayland compositor that is also an RDP server? or maybe they're all like that already?
Also note that there is a critical difference in how they talk to the OS:
* VNC (and other non-RDP solutions like TeamViewer etc): fully independent application, does not change how Windows works because it's effectively just an interactive screen recorder running for your user account.
* RDP: is an actual Windows remote user session that hijacks the computer (so a local user can't see what's happening) and hooks directly into Windows with its own device bindings and login properties (e.g, you can't just click start -> shut down, instead you need to command-line your way to victory).
If you want to remote into a machine that's playing audio without interfering with that, RDP is flat out not an option. Even if you pick "leave audio on the remote", the fact that RPD forces windows to use a different audio device is enough to interfere with playback.
RDP doesn't need to tie into the OS like that. Plenty of ways to run X11 over RDP, for instance. And unlike in VNC, you can actually use the forward/back buttons on your mouse!
RDP in Windows happens to be implemented using some fancy tricks that make it a much better OS for remote work than any Linux distro, but that doesn't mean that's the only possible implementation. Whatever logic can be used to detect block updates in VNC works just as well over RDP. Audio over RDP also works fine on both Windows and Linux so I don't see what the problem would be anywhere else.
As for the shutdown thing, Linux seems to do that too. Makes sense if you use your computer as a terminal server, I guess. I don't reboot my computer over RDP enough to care, really. Still, that's just an implementation choice, nothing to do with the protocol itself.
So -
* VNC: Low overhead / Low fidelity
* RDP (and other remote desktop protocols, e.g. Frame Remoting Protocol, Horizon Blast, Citrix ICA/HDX): Higher overhead / High fidelity