Merge branch 'main' into add-llm-medical-eval

adap · Sep 9, 2024 · 2b72996 · 2b72996
2 parents 822cf4d + 9afe0f8
commit 2b72996
Show file tree

Hide file tree

Showing 3 changed files with 2 additions and 4 deletions.
diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.yml
@@ -146,8 +146,6 @@ jobs:
         if: ${{ github.repository == 'adap/flower' && !github.event.pull_request.head.repo.fork && github.actor != 'dependabot[bot]' }}
         run: |
           python -m pip install https://${{ env.ARTIFACT_BUCKET }}/py/${{ needs.wheel.outputs.dir }}/${{ needs.wheel.outputs.short_sha }}/${{ needs.wheel.outputs.whl_path }}
-      - name: Install e2e components
-        run: pip install .
       - name: Download dataset
         if: ${{ matrix.dataset }}
         run: python -c "${{ matrix.dataset }}"

diff --git a/benchmarks/flowertune-llm/evaluation/README.md b/benchmarks/flowertune-llm/evaluation/README.md
@@ -37,7 +37,7 @@ The default template generated by `flwr new` (see the [Project Creation Instruct
 
 |            | MBPP  | HumanEval | MultiPL-E (JS) | MultiPL-E (C++) |  Avg  |  
 |:----------:|:-----:|:---------:|:--------------:|:---------------:|:-----:|
-| Pass@1 (%) | 32.60 |   26.83   |     29.81      |      24.22      | 28.37 |
+| Pass@1 (%) | 31.60 |   23.78   |     28.57      |      25.47      | 27.36 |
 
 
 ## Make submission on FlowerTune LLM Leaderboard

diff --git a/benchmarks/flowertune-llm/evaluation/general-nlp/README.md b/benchmarks/flowertune-llm/evaluation/general-nlp/README.md
@@ -23,7 +23,7 @@ huggingface-cli login
 Download data from [FastChat](https://github.com/lm-sys/FastChat):
 
 ```shell
-git clone --depth=1 https://github.com/lm-sys/FastChat.git && cd FastChat && git checkout d561f87b24de197e25e3ddf7e09af93ced8dfe36 && mv fastchat/llm_judge/data ../data && cd .. && rm -rf FastChat
+git clone https://github.com/lm-sys/FastChat.git && cd FastChat && git checkout d561f87b24de197e25e3ddf7e09af93ced8dfe36 && mv fastchat/llm_judge/data ../data && cd .. && rm -rf FastChat
 ```