# Databricks benchmark finds declarative Spark pipelines cut agent cluster time 34x

> Original title: Built for Agents: The Case for Spark Declarative Pipelines

- Source: Databricks Blog
- Published: 2026-10-23T19:00:00.000Z
- Bad Influence: https://badinfluence.news/items/j0bmfmudv73lo57zgjoobh42k
- Original: https://www.databricks.com/blog/built-agents-case-spark-declarative-pipelines

## Summary

Databricks benchmarked AI agents writing Spark pipelines procedurally and declaratively on 22 tasks with 12 datasets each, using Opus 4.8. Declarative builds used about 34x less executor time, about half the lines of code, and 2.3x more LLM tokens. Silent logic errors were similar between the two approaches, at 27.7% procedural and 32.2% declarative.

## Why it matters

The benchmark compares procedural and declarative pipeline builds on the same agent and tasks, showing where cost, code size, and silent errors differ.