v0.9.0.13
What's Changed
garak's 1st birthday! π
Headlines in this birthday release:
- Multimodal support! LLaVa + FigStep - HUGE thanks to @DavidLee528
- LiteLLM generator support - thanks to @Tien-Cheng!
- DoNotAnswer probe for prompts an LLM should decline - thanks to @AhsanAyub!
- LangChain Serve generator support - thanks to @GustavFredrikson!
- Support for NIM generators, NVIDIA Inference Microservices
- OpenAI abstraction with parallelisation support
- Windows compatibility enhanced, with optional testing in the workflow
- Hugging Face speedup
plugins
- LiteLLM generator by @Tien-Cheng in #572
- Support for using 'langchain serve' endpoints by @GustavFredrikson in #588
- Enable parallel OpenAI calls by @jmartin-tech in #645
- Multi-modal Jailbreaking Attack on LLaVA by @DavidLee528 in #587
- bump openai module version to match paths in latest litellm by @leondz in #664
- generator: NIM by @leondz in #637
- Probe: Do Not Answer by @AhsanAyub in #608
architecture
- change supported pythons to 3.10-3.12 by @leondz in #503
- add more detailed TAP docs by @leondz in #504
- add multiple-result aggregator by @leondz in #505
- add post buff hook by @erickgalinkin in #506
- Add fleshed-out docs to all probes by @leondz in #507
- add bibtex under citation info in readme by @leondz in #511
- define broad test, all probes, 1 gen per by @leondz in #514
- Feature/taxonomy payloads by @leondz in #519
- include paraphrasing in broad conf by @leondz in #521
- choose whether buffing will also include the original prompt by @leondz in #523
- add config var for capping max # buffed prompts to add per buff by @leondz in #526
- document Probe.probe(); skip a buff hook if no buffs by @leondz in #527
- add type hints to base.Probe; fix base probe rst by @leondz in #528
- Bump datasets package by @shubhobm in #536
- Add ConversationalPipeline for huggingface models by @erickgalinkin in #539
- add generator for supporting openai module v0.x by @leondz in #553
- Update README.md by @erickgalinkin in #558
- Minor typo in FAQ by @jmartin-tech in #562
- Add additional error message when doc is None type by @DavidLee528 in #566
- shared constant & string literal by @jmartin-tech in #571
- Spelling corrections for multiple locations by @jmartin-tech in #564
- Reduce Huggingface GPU utilization by @erickgalinkin in #567
- skip
verbose
flag in secondary parser by @jmartin-tech in #576 - Added project twitter link and corrected a grammatic error by @codebrain001 in #578
- Convert GGML to expect GGUF format by @jmartin-tech in #581
- Update workflows: CLA asst bump, PR & manual testing by @leondz in #591
- add test de-duping using skip-duplicate-actions by @leondz in #597
- Remove
#!
entries from files not intended as executables by @jmartin-tech in #612 - Further align shebangs with code that has executable entry points by @leondz in #613
- interactive mode intro by @leondz in #614
- add tests for
ggml
generator by @jmartin-tech in #618 - add var for generator context_len and populate this for some generators by @leondz in #616
- allow generators.Base.generate() to take an optional param specifying generation count by @leondz in #600
- Enable windows tests as github action by @jmartin-tech in #626
- add on-demand macos testing by @leondz in #631
- macOS test install from correct path by @jmartin-tech in #633
- consolidate test file cleanup by @jmartin-tech in #634
- bump discord link by @leondz in #648
- meta the arguments a bit for GET vs other request types by @jmartin-tech in #640
stability
- fail gracefully if nvcf rejects input; compact zalgo prompts by @leondz in #509
- log & skip past NVCF 4xx errors by @leondz in #533
- fix empty autodan prompts & poor detector behaviour by @leondz in #534
- Fix AutoDAN issues by @erickgalinkin in #537
- fix bad nonetype handling in atkgen probe by @leondz in #538
- Division by zero error fixed in HTML report generation by @CoderMayhem in #545
- cap cohere lib version by @leondz in #569
- rm deprecated model from example by @leondz in #575
- Attack fixes by @erickgalinkin in #555
- More regex as string literal by @jmartin-tech in #586
- Bugfix/action dedupe by @leondz in #598
- wrap cli exec to gracefully catch keyboard exit signal by @jmartin-tech in #603
- Enforce warning output for
garak
classes. by @jmartin-tech in #605 - bump hf transformers v to avoid transformers#30076 by @leondz in #636
- update avidtools to remove typing reference by @jmartin-tech in #639
- torch v bump by @leondz in #649
- Pause FigStepTiny by @leondz in #652
- Bugfix/visual jailbreak pause by @leondz in #653
- limit push test to main by @jmartin-tech in #661
- Update MANIFEST.in so all resources are installed by @JKL98ISR in #660
- handle extant but closed
hitlogfile
file by @leondz in #665
New Contributors
- @CoderMayhem made their first contribution in #545
- @jmartin-tech made their first contribution in #562
- @codebrain001 made their first contribution in #578
- @Tien-Cheng made their first contribution in #572
- @GustavFredrikson made their first contribution in #588
- @JKL98ISR made their first contribution in #660
- @AhsanAyub made their first contribution in #608
Full Changelog: v0.9.0.12...v0.9.0.13