Skip to content
Trending storyDeveloping

SanSi loops a 1.4B model's layers to improve decision accuracy

1 article1 sourcesince Oct 13Last article 2h ago ·

Overview

AISummary of 1 article

SanSi reruns the layers of Ouro-1.4B up to 8 times and reads option probabilities after each loop, so it can stop early or keep looping, according to a paper described by Rohan Paul on X.

On 10,027 test decisions, SanSi scored 72.0% against 58.4% for a same-shape model without loops, using 7.7 times the compute. On liar chains longer than any seen in training, it reached 72.6%, while a 4.2B single-pass model scored 50.0%, the chance level.

Written by AI from the articles below · updated Oct 13, 4:31 PM ET

Check the sources:

Latest developmentOct 13, 4:02 PM ET
SanSi loops a 1.4B model to beat same-size single-pass decisions

Article timeline

The articles in this story. Times are ET.

Oct 13
  1. Rohan PaulX
    SanSi loops a 1.4B model to beat same-size single-pass decisions

    AISanSi reruns the layers of Ouro-1.4B up to 8 times and reads option probabilities after each loop, so it can stop early or run longer. On 10,027 test decisions it scored 72.0% against 58.4% for a same-shape model without loops, at 7.7x the compute. On liar chains longer than its training data, it reached 72.6% while a 4.2B single-pass model scored 50.0%, chance level.

    Image from @rohanpaul_ai's post

Heat trend

Not enough continuous observations to show a trend yet.