Compare the sparse model on downstream tasks

[iidsample]

Hello Authors,

We have been trying to replicate the results for downstream tasks, based on details here.

However, we are unable to figure out how to specify the sparse-opt-model to run.
Being unfamiliar with the code it looks like the current sparse-opt-model does not seem to have integration with the code in Decentralized_FM_alpha.
Can you please provide additional information on how can we reproduce the down stream task numbers.

[bilgeacun]

@lzcemma can you help with this please?

[lzcemma]

Hi,
Step 1: We use lm-eval-harness to generate test data and save them in json file, as shown in this file

Step 2: Once you have the generated file, simply replace the file path in the script example.

Step 3: The script will automatically save generated test samples in the output path. Please use this file to run evaluate.