I've played with a 4.3" Galaxy S2 recently (I have a 3.2" phone right now), and I it didn't feel large at all to me. It seemed pretty normal. I could probably go for one that is a bit bigger than that.

Also the "theory" that everything above 3.5" is too big for one-handed actions is completely untrue. I have average hands, and I could easily use the GS2 with one hand, even up to the top.