[BUG] AttributeError: 'NoneType' object has no attribute 'builder_configs'

[roaminwind]

Describe the bug

Kept reporting Error.
I uninstall and install but it doesn't work. I don't know what happened.
Something below is what I think counts.
│ /root/autodl-tmp/openr1/lib/python3.11/site-packages/lighteval/tasks/extended/lcb/main.py:118 in │
│ │
│ 115 │
│ 116 extend_enum(Metrics, "lcb_codegen_metric", lcb_codegen_metric) │
│ 117 │
│ ❱ 118 configs = get_dataset_config_names("livecodebench/code_generation_lite", trust_remote_co │
│ 119 │
│ 120 tasks = [] │
│ 121 │
│ │
│ /root/autodl-tmp/openr1/lib/python3.11/site-packages/datasets/inspect.py:175 in get_dataset_config_names │
│ │
│ 172 │ │ **download_kwargs, │
│ 173 │ ) │
│ 174 │ builder_cls = get_dataset_builder_class(dataset_module, dataset_name=os.path.basenam │
│ ❱ 175 │ return list(builder_cls.builder_configs.keys()) or [ │
│ 176 │ │ dataset_module.builder_kwargs.get("config_name", builder_cls.DEFAULT_CONFIG_NAME │
│ 177 │ ] │
│ 178 │
╰─────────────────────────────────────────────────────────────────────
AttributeError: 'NoneType' object has no attribute 'builder_configs'

To Reproduce

make evaluate MODEL=deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B TASK=aime24 PARALLEL=data NUM_GPUS=3

Expected behavior

I just want that run without error :<

Version info

lighteval 0.8.1
3*A100
cuda12.4
my uv virtual env is python3.11
accelerate 1.4.0
datasets 3.5.0
deepspeed 0.15.4
huggingface-hub 0.30.2
llvmlite 0.44.0
ninja 1.11.1.4
nltk 3.9.1
pip 25.0.1
torch 2.6.0
torchaudio 2.6.0
torchvision 0.21.0
tqdm 4.67.1
transformers 4.51.2
triton 3.2.0
trl 0.17.0.dev0
vllm 0.8.3
wandb 0.19.9

[roaminwind]

I found the same problem at #595
but by upgrading the problem still exists.
and the version is still 0.8.1

[NathanHB]

hey ! should be fixed in #677
this happens because if the hub is down then the repo can be downloaded and lighteval crashes, i've hardcoded the configs so that it does not happen.