# Together AI explains how auto repair turns GPU faults into healthy nodes

> Original title: Auto repair for Together GPU Clusters: from GPU fault to healthy node

- Source: Together AI Blog
- Published: 2026-10-27T00:00:00.000Z
- Bad Influence: https://badinfluence.news/items/noxuopxz4sju8rovre6pougef
- Original: https://www.together.ai/blog/auto-repair-for-together-gpu-clusters

## Summary

Together AI describes how its auto repair system handles GPU cluster faults, such as Xid errors, ECC errors and InfiniBand link flapping. Detection runs both inside the VM and on the host, and each fault maps to the lightest repair action, with customers choosing whether repairs wait for approval or run automatically. Repairs escalate from reboot to reprovision to migration when a fault recurs, and migrated machines are re-validated automatically before returning to the pool.

## Why it matters

The post walks through each stage of GPU node auto-repair, from detection and policy through escalation to re-validation, showing how the design avoids unsafe mass replacements.