This is all true. Additionally, the payback from optimizing purely scalar arithmetic harder has gone down more and more over time compared to almost anything else.

For example, eliminating an extra load or store is often worth more than eliminating 100 extra arithmetic operations these days.