Replication of Still 1.5 Preview AIME: 32% AIME not 39%

[michaelzhiluo]

I have ran Still 1.5 Pass@16 averaged and got 31% Pass 1 averaged over 16 passes of AIME, temperature 0.6 (like in deepseek) and top_p=0.95 (like in deepseek).

Are there other parameters that need to be tuned to replicate these results?

[michaelzhiluo]

Update: We tried 32K (max context length for qwen) and did it over pass 16:

Total problems: 30
Pass@1 average over 16 runs: 32.71%
Pass@16: 73.33%