I've found overcommit useful for some scientific computing applications, but you're right it's not the most intuitive default. You can disable memory overcommit on Linux for the same behavior as Windows, if you want.
Disabling overcommit on Linux doesn't tend to enable library and utility developers to have used memory responsibly. So it can be hard to run a system with common software.
I keep it disabled on one of my systems for exactly that reason: I want to make sure my own code works with overcommit disabled, as it'll always be on Windows or OpenBSD.
https://www.kernel.org/doc/Documentation/vm/overcommit-accou...
Also, I found this post that suggests Windows technically does overcommit memory, but only for stacks(‽): https://superuser.com/questions/1194263/will-microsoft-windo...