You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: README.md
+4-4
Original file line number
Diff line number
Diff line change
@@ -128,7 +128,7 @@ There are three main ways to use PromptWizard:
128
128
- Both the train and test set follow the same format
129
129
- Every sample in the ```.jsonl``` should have 2 fields :
130
130
1) ```question``` : It should contain the complete question that is to asked to the LLM
131
-
2) ```answer``` : It should contain the ground truth answer which can be verbose or consize
131
+
2) ```answer``` : It should contain the ground truth answer which can be verbose or concise
132
132
133
133
134
134
### Run on Custom Dataset
@@ -177,10 +177,10 @@ NOTE : Refer to [demos](demos) folder for examples of folders for four datasets.
177
177
- ```generate_synthetic_examples``` is a global hyperparameter which can be used when there are no training samples and we want to generate synthetic data for training
178
178
- ```use_examples``` is a global hyperparameter which can be used to optimize prompts using training data
179
179
3) Create a dataset specific class which inherits ```class DatasetSpecificProcessing``` similar to ```GSM8k(DatasetSpecificProcessing)``` in [demo.ipynb](demos/gsm8k/demo.ipynb) and define the following functions in it
180
-
1) In ```def extract_answer_from_output()``` : This is a dataset specific function, given the ```answer``` from the dataset it should extract and return a consize form of the answer. Note that based on the dataset it can also simply return the ```answer``` as it is like in case of SVAMP and AQUARAT datasets
181
-
2) ```def extract_final_answer()``` : This is a LLM output specific function, given the verbose answer from the LLM it should extract and return the consize final answer
180
+
1) In ```def extract_answer_from_output()``` : This is a dataset specific function, given the ```answer``` from the dataset it should extract and return a concise form of the answer. Note that based on the dataset it can also simply return the ```answer``` as it is like in case of SVAMP and AQUARAT datasets
181
+
2) ```def extract_final_answer()``` : This is a LLM output specific function, given the verbose answer from the LLM it should extract and return the concise final answer
182
182
3) Define ```def access_answer()``` : This function takes an input the LLM output, then does the following:
183
-
- Extracts the consize answer using ```def extract_final_answer()``` from the LLM output as defined above
183
+
- Extracts the concise answer using ```def extract_final_answer()``` from the LLM output as defined above
184
184
- Evaluates the extracted answer with the ground truth and retuns
185
185
- Extracted answer from LLM output
186
186
- Boolean value indicating if answer is correct or not
0 commit comments