Databricks Lakebase branching gives CI/CD pipelines per-PR Postgres test databases
Overview
Databricks describes how Snorkel AI uses Lakebase branching to give each pull request its own copy of its roughly 1 TB production Postgres database.
Snorkel reports that provisioning fell from about 20 minutes to under 60 seconds at p95. The main branch is kept in sync with production through AWS DMS, and each PR gets a branch that expires on a TTL. CI authenticates with service principals and short-lived OAuth tokens.
Lakebase is Databricks' serverless Postgres database. The source for these figures is Databricks' own blog post, so the performance numbers are Databricks' and Snorkel's reported results, not independent measurements.
Written by AI from the articles below · updated Oct 31, 6:10 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- Databricks BlogOfficialPickDatabricks details how Snorkel uses Lakebase branching for per-PR test databases
AISnorkel AI used Lakebase branching to give each pull request its own copy of its roughly 1 TB production Postgres database, cutting provisioning from about 20 minutes to under 60 seconds at p95. The setup keeps the main branch synced with production via AWS DMS, creates a TTL-bound branch per PR, and authenticates CI with service principals and short-lived OAuth tokens.
Heat Trend
Not enough continuous observations to show a trend yet.