Skip to content

Commit

Permalink
update eval bash and result
Browse files Browse the repository at this point in the history
  • Loading branch information
ZiyueXu77 committed Sep 20, 2024
1 parent 8947b9a commit b8cbf6d
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion examples/advanced/rag/embedding/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,6 @@ NLI | 0.7586 | 0.7895 | 0.8033 | 0.8045
Squad | 0.8206 | 0.8154 | 0.8051 | 0.8042
Quora | 0.8161 | 0.8121 | 0.7891 | 0.7854
All | 0.8497 | 0.8523 | 0.8426 | 0.8384
Federated | 0.8444 | 0.8368 | 0.8269 | 0.8246
Federated | 0.8443 | 0.8367 | 0.8261 | 0.8249

As shown, the federated training results are better than individual site's, and can be close to the centralized training results, demonstrating the effectiveness of NVFlare in embedding model tuning tasks.
2 changes: 1 addition & 1 deletion examples/advanced/rag/embedding/eval_all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ do
done

echo "Evaluation on model federated"
python utils/eval_model.py --model_path /tmp/embed/nvflare/workspace/site-1/simulate_job/app_site-1/models/mpnet-base-nli/global
python utils/eval_model.py --model_path /tmp/embed/nvflare/workspace_api/site-1/models/mpnet-base-nli/global

0 comments on commit b8cbf6d

Please sign in to comment.