The constitution says the president must "take Care that the Laws be faithfully executed". That means hiring people to enforce the laws, that's the civil servant system.
That means the whole Executive branch is the president and if he can delegate his power to some other people he also should be able to revoke the delegation and fire those people, which is not the case now. The president is semi-successful in firing these "servants" and some judges insist that it's illegal.