File tree 1 file changed +5
-3
lines changed
1 file changed +5
-3
lines changed Original file line number Diff line number Diff line change @@ -31,9 +31,11 @@ SciCode sources challenging and realistic research-level coding problems across
31
31
32
32
| Models | Main Problem Resolve Rate | <span style =" color :grey " >Subproblem</span > |
33
33
| --------------------------| -------------------------------------| -------------------------------------|
34
- | 🥇 OpenAI o1-preview | <div align =" center " >** 7.7** </div > | <div align =" center " style =" color :grey " >28.5</div > |
35
- | 🥈 Claude3.5-Sonnet | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >26.0</div > |
36
- | 🥉 Claude3.5-Sonnet (new) | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >25.3</div > |
34
+ | 🥇 OpenAI o3-mini | <div align =" center " >** 9.2** </div > | <div align =" center " style =" color :grey " >33.0</div > |
35
+ | 🥈 OpenAI o1-preview | <div align =" center " >** 7.7** </div > | <div align =" center " style =" color :grey " >28.5</div > |
36
+ | 🥉 Deepseek-R1 | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >28.5</div > |
37
+ | Claude3.5-Sonnet | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >26.0</div > |
38
+ | Claude3.5-Sonnet (new) | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >25.3</div > |
37
39
| Deepseek-v3 | <div align =" center " >** 3.1** </div > | <div align =" center " style =" color :grey " >23.7</div > |
38
40
| Deepseek-Coder-v2 | <div align =" center " >** 3.1** </div > | <div align =" center " style =" color :grey " >21.2</div > |
39
41
| GPT-4o | <div align =" center " >** 1.5** </div > | <div align =" center " style =" color :grey " >25.0</div > |
You can’t perform that action at this time.
0 commit comments