How do you know you have to go back and audit except by looking at the forgeable vote counting results? Those results could easily be hacked to not meet whatever the criteria are for an audit.

Hand counted paper ballots are the only way to be sure, and elections are important enough to be worth it.