I've had a similar experience with my iPhone 5 as well. It was significantly worse during the beginning of the beta builds, however now it seems to be better than iOS6 (for me).

I'm guessing it's something similar to you – a wonky app draining my battery that is handled better on 7.