Indeed, this is a deceptively really hard problem that I think nobody perfectly solved yet. The main problem with the deep learning route is it being resource demanding (both computation and memory expensive). Hopefully these problems will automatically go away in a couple of years as the mobile devices become more powerful and the deep learning architecture gets more light-weighted.