Skip to content
View original post on X: roonX· 42/100AI score42/100

roon argues a slowdown should shift AI labs' focus to alignment and control research

AISummary

roon (@tszzl) says a slowdown would be most useful if channeled into alignment, mechanistic interpretability, monitorability, and decision theory rather than general capability gains.

He argues AI labs already face enormous pressure to solve alignment and control, since confidence that the next training run is safe is their top priority.

He adds that automated alignment research is possible and that major AI companies could redirect their focus to it under pressure from capital markets.

Post on XView on X
roonVerified on X
@tszzl

"what would be done during the slowdown" is a good question, because i think people imagine beaurucrat-regulators sitting around on useless subcommittees and wasting the whole thing on wordcel slop analysis. in fact if you asked me two years ago, i would have suggested that is exactly what would happen.

but these are scary times, and they call for us to think lofty & even impractical thoughts, that a 'sensible' person might have dismissed a few quarters ago. crises change the world overnight: the HFI has shifted something about the culture, broadly

this is something obvious that i missed, which is that the ai labs and the markets more generally just have just unbelievable pressure riding on the solving of alignment & control. even now, marginal improvements in alignment, safety, and model behavior are worth tremendous value to the labs. being able to convince themselves that the next training run is safe is the priority zero, one, and two in terms of capabilities progress right now. it's possible that will change and political will dissipates if some sense of comfort re-emerges after a period of relative safety (which would be a bad development; paranoia must be maintained through any potential periods of ai development where the models are too smart to reveal their behavioral predilections under scrutiny)

if there was a slowdown, it would ideally be channeled into some sort of tradeoff in which you can narrowly improve capabilities that prioritize risk-adjusted-return of alignment skills, mechinterp, monitorability, decision theory, behavioral generalization, etc. progress on all these areas is highly possible; automated alignment research is possible. the structure of the businesses may have to change, but all the trillion dollar ai behemoths of the world would shift their focus to doing this and only this at the behest of trillions of dollars in capital markets

Source: roon · x.comPublished