Skip to content
View original post on X: Gergely Orosz
Published
39/100

Inference vendors' value rests on scarce GPUs, infra ease, and data security

AIGergely Orosz says inference vendors sell three things: access to hard-to-get GPUs, infrastructure abstraction that simplifies operations, and privacy and security of customer data. He argues a data breach would normally be very damaging, but GPU scarcity leaves customers with few alternatives.
Gergely OroszVerified on X
@GergelyOrosz
View on X

Most important stuff inference vendors offer:

1. Access to GPUs (hard to get!)

2. Infra abstraction so it's easier to operate

3. Privacy of data and security

Usually, a breach would be very bad. BUT GPUs are so scarce today that customers don't have all that many options...

Stephanie Palazzolo@steph_palazzolo
Fireworks, the most highly-valued out of a growing group of inference providers, told customers including Harvey that it had been hit with a hack that exposed customer data. Some customers are now considering moving their business off the platform. https://www.theinformation.com/articles/new-fireworks-hack-threatens-1-billion-business
View quoted post on X
Source details

Gergely Orosz · x.comPublished · added here

Original title: Most important stuff inference vendors offer:

Topics

Deployment & engineeringExpert opinion