You are 100% right. Though this problem isn't unique to India by any means. In fact you often hear of sexual assault cases re. taxis and ride share going through the courts in Australia.
That said I've personally witnessed things "nearly" happen, and heard some harrowing tales besides, to / from lone women travelers in a number of countries throughout Asia, the Middle East and Europe.
The world is not a safe place for women.
As an aside, Singapore seems to be one of the outliers here. We saw a woman exercising at about midnight one night coming from the airport. Taxi driver told us it wasn't unusual, ostensibly thanks to capital punishment. No way you'd feel comfortable doing that in Australia.
You could be possibly sexually assaulted, trafficked, or other things.