v1.0.0

What's Changed

• chore: Update readme with installation tips by @danielezhu in #181
• fix: readme by @polaschwoebel in #196
• docs: add troubleshooting item for OOM errors by @keerthanvasist in #198
• fix: add data for example notebook by @polaschwoebel in #203
• fix: update terminology in README and source code by @danielezhu in #208
• feat: implement Transform and TransformPipeline classes for modular redesign by @danielezhu in #209
• feat: implement helper models used by evaluation algorithms by @danielezhu in #210
• feat: implement transforms for summarization accuracy metrics by @danielezhu in #211
• docs: update README to include information about Windows support by @danielezhu in #213
• fix: update the default prompt templates for the built-in datasets by @jmikko in #212
• feat: update implementation of SummarizationAccuracy to use Transform-based approach by @danielezhu in #214
• feat: implement transforms for semantic perturbations by @danielezhu in #215
• refactor: update Transform API by @danielezhu in #216
• feat: add prompt template to report by @oyangz in #217
• feat: update various transforms to accept multiple input keys by @danielezhu in #218
• chore: change PromptComposer.PLACEHOLDER from "feature" to "model_input" by @danielezhu in #219
• feat: update GetModelResponse transform to support multiple model invocations on the same input by @danielezhu in #220
• feat: update implementation of GeneralSemanticRobustness to use Transform-based approach by @danielezhu in #222
• fix: update GetModelResponse transform to work with any ModelRunner by @danielezhu in #228
• fix: restore semantic perturbation constants to their original values by @danielezhu in #229
• feat: example notebook for comparative plotting by @polaschwoebel in #223
• refactor: move repeated code in evaluate method into util functions and simplify the EvalAlgorithmInterface method signatures by @danielezhu in #224
• feat: updated docstrings by @polaschwoebel in #225
• chore: restore evaluate_sample and evaluate signatures in EvalAlgorithmInterface by @danielezhu in #231
• refactor: update evaluate_dataset to take in a dataset instead of dataset config by @danielezhu in #232
• feat: update implementation of SummarizationAccuracySemanticRobustness to use Transform-based approach by @danielezhu in #233
• feat: update implementation of QAAccuracy to use Transform-based approach by @danielezhu in #234
• feat: update implementation of QAAccuracySemanticRobustness to use Transform-based approach by @danielezhu in #235
• feat: update implementation of ClassificationAccuracy to use Transform-based approach by @danielezhu in #236
• feat: update implementation of ClassificationAccuracySemanticRobustness to use Transform-based approach by @danielezhu in #237
• Updating third party attributions by @malhotra18 in #239
• feat: update implementation of FactualKnowledge to use Transform-based approach by @danielezhu in #238
• feat: update implementation of PromptStereotyping to use Transform-based approach by @danielezhu in #240
• fix: set default region for boto3 client to access built-in datasets by @oyangz in #242
• feat: update implementation of Toxicity to use Transform-based approach by @danielezhu in #241
• build: bump fmeval version to 1.0.0 by @danielezhu in #243

New Contributors

• @jmikko made their first contribution in #212

Full Changelog: v0.4.0...v1.0.0