Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feat] LiveBench 2409 #304

Merged
merged 6 commits into from
Oct 7, 2024
Merged

[Feat] LiveBench 2409 #304

merged 6 commits into from
Oct 7, 2024

Conversation

pufanyi
Copy link
Collaborator

@pufanyi pufanyi commented Oct 7, 2024 

accelerate launch -m lmms_eval --model=llava --model_args=pretrained=liuhaotian/llava-v1.5-7b --batch_size=1 --log_samples --tasks=live_bench_2409 --output_path=./logs/

@pufanyi pufanyi requested a review from Luodian October 7, 2024 17:24
@Luodian Luodian merged commit b838ff6 into main Oct 7, 2024
2 checks passed
pufanyi added a commit that referenced this pull request Oct 9, 2024
@pufanyi
Squashed commit of the following:
1dd348a 
commit 581a672
Author: Yinan He <[email protected]>
Date:   Wed Oct 9 06:11:24 2024 +0200

    [Fix] Fix cache_dir issue where MVBench cannot be found (#306)

    * [add] add internvideo2 support && change mvbench to video branch

    * [add] answer_prompt of internvideo2

    * [add] change video type of internvideo2

    * [fix] update template of mvbench

    * [reformat]

    * [fix] generate_until_multi_round

    * [Feat] videochat2 support

    * [feat] Link cache_path to cache_dir if no unzip or untar

    * [feat] new variable in dataset kwargs: create_link

    ---------

    Co-authored-by: heyinan <[email protected]>

commit 99fcd39
Author: GIO PAIK <[email protected]>
Date:   Tue Oct 8 15:56:53 2024 +0900

    fix: Invalid group in mmsearch.yaml (#305)

commit 6e1d747
Author: Li Bo <[email protected]>
Date:   Tue Oct 8 08:42:20 2024 +0800

    [Doc] add more detailed task guide to explain the variables in yaml configuration file (#303)

commit b838ff6
Author: Pu Fanyi <[email protected]>
Date:   Tue Oct 8 01:25:35 2024 +0800

    [Feat] LiveBench 2409 (#304)

    * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

    * Refactor DefaultWebsite class in website.py

    * Refactor dataset name and limit processed images to 2

    * Refactor extract_infomation.py and update prompts

    * update

    * Refactor live_bench_2409.yaml and live_bench.yaml

commit 09d15c3
Author: Zhijian Liu <[email protected]>
Date:   Sat Oct 5 11:46:44 2024 -0400

    Fix MMMU-Pro evaluation (#296)
kcz358 pushed a commit that referenced this pull request Oct 9, 2024
@pufanyi
[Fix] LiveBench 2409 (#308)
39e1586 
* Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

* Refactor DefaultWebsite class in website.py

* Refactor dataset name and limit processed images to 2

* Refactor extract_infomation.py and update prompts

* update

* Refactor live_bench_2409.yaml and live_bench.yaml

* Refactor live_bench/utils_v2.py and question_finalizer.py

* Squashed commit of the following:

commit 581a672
Author: Yinan He <[email protected]>
Date:   Wed Oct 9 06:11:24 2024 +0200

    [Fix] Fix cache_dir issue where MVBench cannot be found (#306)

    * [add] add internvideo2 support && change mvbench to video branch

    * [add] answer_prompt of internvideo2

    * [add] change video type of internvideo2

    * [fix] update template of mvbench

    * [reformat]

    * [fix] generate_until_multi_round

    * [Feat] videochat2 support

    * [feat] Link cache_path to cache_dir if no unzip or untar

    * [feat] new variable in dataset kwargs: create_link

    ---------

    Co-authored-by: heyinan <[email protected]>

commit 99fcd39
Author: GIO PAIK <[email protected]>
Date:   Tue Oct 8 15:56:53 2024 +0900

    fix: Invalid group in mmsearch.yaml (#305)

commit 6e1d747
Author: Li Bo <[email protected]>
Date:   Tue Oct 8 08:42:20 2024 +0800

    [Doc] add more detailed task guide to explain the variables in yaml configuration file (#303)

commit b838ff6
Author: Pu Fanyi <[email protected]>
Date:   Tue Oct 8 01:25:35 2024 +0800

    [Feat] LiveBench 2409 (#304)

    * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

    * Refactor DefaultWebsite class in website.py

    * Refactor dataset name and limit processed images to 2

    * Refactor extract_infomation.py and update prompts

    * update

    * Refactor live_bench_2409.yaml and live_bench.yaml

commit 09d15c3
Author: Zhijian Liu <[email protected]>
Date:   Sat Oct 5 11:46:44 2024 -0400

    Fix MMMU-Pro evaluation (#296)

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Luodian Luodian Luodian approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@pufanyi @Luodian