It's not for performance but for not having to permanently store terabytes of differently encoded data.

You could possibly also pre-populate cache - all new media files get sent to encoder and cached preemptively, and if they are popular enough they just stay there till they are not.