It's normal for police at a traffic stop to take your license back to their car while they write a ticket or whatever. Until laws change, having your only license on your phone means handing your phone to an officer until they are satisfied they no longer need it.
States that have implemented mobile drivers licenses are starting to issue handheld readers to police officers, precisely so what you describe doesn't happen.
The people building this know nobody wants to hand their phone over to the police.