Stanford and University of Washington researchers devised a technique to create a new AI model dubbed “s1.”
They have already open-sourced it onGitHub, along with the code and data used to build it.
A paperpublishedlast Friday explained how the team achieved these results through clever technical tricks.
They extracted the reasoning capabilities from one of Google’s AI models specifically, Gemini 2.0 Flash Thinking Experimental.
They then trained the base model to mimic its step-by-step problem-solving process on a small dataset.
Others have used this approach before.
In fact, distillation is what OpenAI wasaccusingDeepSeek of doing.
However, the Stanford/UW team found an ultra-low-cost way to implement it through “supervised fine-tuning.”
This process involves explicitly teaching the model how to reason using curated examples.
Their full dataset consisted of only 1,000 carefully selected questions and solutions pulled from Google’s model.
TechCrunch notes that thetrainingprocess took 30 minutes, using 16 Nvidia H100 GPUs.
The researchers also discovered a neat trick to boost s1’s capabilities even further.
They instructed the model to “wait” before providing its final answer.
This command allowed it more time to check its reasoning to arrive at slightly improved solutions.
The model is not without its caveats.
There is also the potential for Google to protest.
It could be waiting to see how OpenAI’s case goes.
