We use land/nitrates/fertilizers to grow crops to feed to cows/pigs/chickens/fish, which convert those plant calories into fewer meat calories. If we ate more plant-based foods, we'd need to grow less, not more.

I understand this is simplistic, as some plants (e.g., iceberg lettuce) provide few calories and are costly to grow. On average, eating from trophic level 1 is more efficient than eating from levels 2 or 3.