The money's in the training, not the inference.

If you look at Apple and Google, they already have their own hardware for inference in their smartphones. They don't need NVidia for that.

Apple and Google use TPUs for training.

https://www.cnbc.com/2024/07/29/apple-says-its-ai-models-wer...

Hmmm, that's worse for NVidia.

NVIDIA owns the interconnects that are used for this training. I’m sure they have their own competing AI accelerator they are working on too.

You don’t need programmability for AI teaining either.