It isn't an either or choice. The people interested in optimizing the performance are typically different people than those interested in implementing syntactic sugar. It is certainly true that growing the overall codebase risks introducing tensions for some feature sets but that is just a consideration you take when diligently adding to the language.
Personally I think it is more crazy that you would optimize 99% of the time just to need it for 1% of the time.