The market for AI inference is important for two reasons.
First, Nvidia seems to havea lock on AI training.
For the time being, this is Nvidia’s market.
Second, the market for AI inference is likely to be much larger than the training market.
Intel’s CEO Pat Gelsinger has a good analogy for this weather models.
Of course, there are two pieces of the inference market cloud and edge.
Cloud inference takes place in the data center and edge inference takes place on the gear.
We have heard people debate the definition of these two recently, the boundaries can get a bit blurry.
Cloud inference is likely to be the most interesting contest to watch.
Nvidia has articulateda very strong casefor why they will transfer their dominance in training to inference.
On the other hand,their bigcompetitorsare going to push very hard for their share of this market.
We expect this to be the center of a lot of attention in coming years.
The market for edge inference is a much more open question.
For starters, no one really knows how much AI models will rely on the edge.
This will greatly reduce the amount of money they have to spend building all those cloud inference data centers.
We suspect that the economics of AI may not pencil out if this is not possible.
We mentioned thatQualcommfaces this problem in smartphones, but the same applies toInteland AMD for PCs.
We have asked everyone about this and have yet to get a clear answer.
This reliance on software begs the question as to how much value there is for semis makers in AI.
Microsoft is kind of an expert at this.
