Inference vendors' value rests on scarce GPUs, infrastructure ease, and data security

Gergely Orosz says inference vendors sell three things: access to hard-to-get GPUs, infrastructure abstraction that simplifies operations, and privacy and security of customer data.

He argues that a data breach would normally be very damaging to a vendor, but GPU scarcity leaves customers with few alternatives.

Orosz's post cites Fireworks, a hack that exposed customer data, as a case where some customers are considering leaving the platform.