Weren't substantial percentages of the US still on subsistence farming/bartering at the time? It's hard to income tax a society without much income.