It is quite large, but I never have trouble fitting it in my pocket (no matter how skinny the jeans). I find it quite comfortable for one-handed texting, and ICS makes texting much easier than earlier releases. Personally I think ICS will make a larger positive impact on your user experience than size could make a negative one.
For one-handed text entry, it works pretty well. There's a reason the keyboard is on the bottom of the screen. But there are non-trivial regions of the screen in the upper corners that cannot be reached by my thumb. And it's occasionally annoying -- the browser tab controls aren't reachable when I hold the phone in my left hand, for example.
That said, the bigger screen is absolutely wonderful for browsing and reading in general; I much prefer Chrome on the GNex to my wife's iPhone 4S.
It's a taste and usage thing. I find for my needs it works very well. Someone who does a lot of one-handed work (while, er, driving maybe) is going to want a smaller phone.
