Real-world benchmarks for tax and IT, as well as a great analysis!
Thrive Holdings publishes real-world AI benchmarks for tax and IT work
AISummary
Boris Power, OpenAI's account on X, praises Thrive Holdings' first real-world benchmarks for tax and IT services. Thrive Holdings, which runs more than 75 accounting and IT services firms, uses the tests to choose which models run its systems. The benchmarks are published at evals.thriveholdings.com.
Post on XView on X
Boris PowerVerified on X
@BorisMPower
Choosing the right AI model is an operating decision. Today, we’re sharing our first set of benchmarks drawn from work across the 75+ accounting and IT services firms that make up Thrive Holdings. We believe the field needs more benchmarks tied to economically valuable work, so builders can understand where models are useful in real-world environments. We use these tests to help us compare model configurations and choose which models our systems run on, with the goal of delivering better service to our customers. We’ll add to the benchmarks as we build new products and expand to additional industries. https://evals.thriveholdings.com/View quoted post on X
Source: Boris Power · x.comPublished