You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Great Work!!! But could you provide some URLs or API endpoints? This would help prevent individuals from submitting unverifiable models, which could potentially compromise the integrity of the leaderboard.
Hi @archersama,
We have a family of new embedding models, CodeXEmbed, and would like to submit its results to our leaderboard. Could you please assist us with this?
{"Model": "CodeXEmbed-7B", "Model Size (Million Parameters)": 7B, "URL": "”, "Apps": 85.22, "CosQA": 33.27, "Synthetic Text2sql":64.57, "CodeSearchNet": 78.84, "CodeSearchNet-CCR": 86.77, "CodeTrans-Contest": 90.64, "CodeTrans-DL": 32.31, "StackOverFlow QA": 94.25, "CodeFeedBack-ST": 80.93, "CodeFeedBack-MT": 57.83, "Avg": 70.46}
{"Model": "CodeXEmbed-2B", "Model Size (Million Parameters)": 2B, "URL": "”, "Apps": 74.99, "CosQA": 36.31, "Synthetic Text2sql":59.00, "CodeSearchNet": 73.50, "CodeSearchNet-CCR": 85.77, "CodeTrans-Contest": 86.63, "CodeTrans-DL": 33.17, "StackOverFlow QA": 90.54, "CodeFeedBack-ST": 81.15, "CodeFeedBack-MT": 53.08, "Avg": 67.41}
{"Model": "CodeXEmbed-400M", "Model Size (Million Parameters)": 2B, "URL": "”, "Apps": 48.57, "CosQA": 34.05, "Synthetic Text2sql":58.96, "CodeSearchNet": 72.53, "CodeSearchNet-CCR": 80.15, "CodeTrans-Contest": 75.67, "CodeTrans-DL": 34.85, "StackOverFlow QA": 89.51, "CodeFeedBack-ST": 78.87, "CodeFeedBack-MT": 45.75, "Avg": 61.89}
The text was updated successfully, but these errors were encountered: