v3.7.0

This release adds the following new features, improvements and bug fixes.

New Features

• Add object detection pipeline (#148)
• Add image caption pipeline (#149)
• Add retrieval task (#150)
• Add no-op pipeline (#152)
• Add new workflow functionality (#155)

Improvements

• Add korean translation to README.md. Thank you @0206pdh! (#138)
• Add links to external articles (#139)
• Update example applications to be consistent (#140)
• Add an article summarization example (#144)
• Add fallback mode for textractor (#145)
• Reorganize pipeline package (#147)
• Update optional package tests to simulate missing packages (#154)
• Add parameter to flatten labels output (#153)
• Update documentation with latest changes (#156)

Bug Fixes

• Fix bug with importing service task when workflow extra not installed (#146)
• Fix inconsistencies with url based tasks (#151)

v3.6.0

This release adds the following new features, improvements and bug fixes.

New Features

• Add post workflow action to API (#129)
• Add tabular pipeline (#134)
• Enhance ServiceTask to support additional use cases (#135)
• Add notebook for tabular pipeline (#136)
• Add topn option to extractor pipeline (#137)

Improvements

• Refactor registering new auto models to use methods in Transformers library (#128)
• Update workflow example application (#130)

Bug Fixes

• No issues this release

v3.5.0

This release adds the following new features, improvements and bug fixes.

New Features

• Add scikit-learn to ONNX export pipeline (#124)
• Add registry methods for auto models (#126)
• Add notebook to demonstrate loading scikit-learn and PyTorch models (#127)

Improvements

• Add parameter to return raw model outputs for labels pipeline (#123)
• Add parameter to use standard pooling for TransformersVectors (#125)

Bug Fixes

• Pass model configuration to ONNX Models (#121)
• Fix incorrect import in Notebooks (#122)

v3.4.0

This release adds the following new features, improvements and bug fixes.

New Features

• Create notebook using extractive qa to build structured data (#117)
• Modify extractor pipeline to support similarity pipeline backed context (#119)

Improvements

• Improve performance of extractor context queries (#120)

Bug Fixes

• Update labels pipeline to filter text classification output (#116)
• Fix issues with Transformers 4.11.2 (#118)

v3.3.0

This release adds the following new features, improvements and bug fixes.

New Features

• Add ONNX export pipeline (#107)
• Add notebook for ONNX pipeline (#108)
• Add ONNX support for Embeddings and Pipelines (#109)
• Support QA models in Trainer pipeline (#111)
• Add notebook for training QA models (#115 )

Improvements

• Remove deprecated packages (#114)

Bug Fixes

• Fix issues with latest Transformers version (#110)

v3.2.0

This release adds the following new features, improvements and bug fixes.

New Features

• Enhance Labels pipeline to support standard text classification models (#95)
• Add Trainer pipeline (#96)
• Modularize txtai install (#97)
• Evaluate if faiss-cpu can be used as default across all platforms (#98)
• Add vector method for sentence-transformers (#101)

Improvements

• Add book search example application (#91)
• Add wiki search example application (#92)
• Change tokenization to default to false for TransformerVectors (#99)
• Infer vector method using path (#100)
• Improve performance when running models through transformers (#102)
• Update notebooks and example applications (#103)

Bug Fixes

• Clear workflow batch during processing bug (#90)

v3.1.0

This release adds the following new features:

• Add support for update/delete embeddings index operations (#86)
• Add Embeddings Cluster component (#87)
• Switch default backend on Windows to Hnswlib (#88)
• Add notebook covering distributed embedding clusters (#89)

v3.0.0

txtai 3.0.0 is a major release with a significant number of new features. This release overhauls the project structure, consolidates logic into pipelines and introduces workflows.

Summary of txtai features:

• 🔎 Large-scale similarity search with multiple index backends (Faiss, Annoy, Hnswlib)
• 📄 Create embeddings for text snippets, documents, audio and images. Supports transformers and word vectors.
• 💡 Machine-learning pipelines to run extractive question-answering, zero-shot labeling, transcription, translation, summarization and text extraction
• ↪️️ Workflows that join pipelines together to aggregate business logic. txtai processes can be microservices or full-fledged indexing workflows.
• 🔗 API bindings for JavaScript, Java, Rust and Go
• ☁️ Cloud-native architecture that scales out with container orchestration systems (e.g. Kubernetes)

New Features

• Add Docker file for API (#59)
• Require Faiss 1.7.0 (#60)
• Add summary pipeline (#65)
• Add text extraction pipeline (#66)
• Add transcription pipeline (#67)
• Add translation pipeline (#68)
• Add workflow framework (#69)
• Add additional pipeline abstraction layer for tensor frameworks (#70)
• Add tests for new v3 functionality (#71)
• Add notebooks covering new v3 functionality (#73)
• Add Pipeline Factory (#76)
• Add API extensions (#77)
• Add workflow builder application (#80)
• Add text segmentation pipeline (#81)
• Add workflow to API (#82)
• Add service workflow task (#83)
• Add object storage workflow task (#84)
• Add URL workflow task (#85)

Improvements

• Refactor code into smaller components and modules (#63)
• Modify pipeline to accept GPU device id (#64)
• Allow direct download of sentence-transformer models (#72)
• Update documentation, add site through GitHub pages (#75)
• Modularize the API (#78)
• Add default truncation to pipelines (#79)

Bug Fixes

• Non intuitive behaviour of Tokenizer (#61)
• [Python 3.9, Mac OS] Code hangs while building embedding index (#62)
• embeddings.index Truncation RuntimeError: The size of tensor a (889) must match the size of tensor b (512) at non-singleton dimension 1 (#74)

v2.0.0

txtai 2.0.0 is a major release with a significant number of new features. This release brings a new zero-shot similarity pipeline, a more streamlined and consistent API, batch support for all modules and integration with Hugging Face Datasets.

In addition to Python, txtai has API support for JavaScript, Java, Rust and Go.

New Features

• [BREAKING CHANGES] Make API definitions consistent (#54)
• Zero-shot similarity pipeline (#21, #49)
• Add batch support for all modules (#18, #53)
• Add example notebook integrating Hugging Face Datasets (#26)
• Add example notebook that adds semantic search to existing system (#57)

Improvements

• Add API tests, increase test coverage (#42)
• Refactor pipeline component (#44)
• Upgrade to Transformers 4.x (#45)
• Review, organize and update example notebooks (#52)
• Allow setting ANN index parameters (#55)
• Modify API add method to stream data (#56)

Bug Fixes

• Fix language support issues (#39, #43)

v1.5.0

This release adds the following enhancements and bug fixes:

• Refresh example notebooks and add notebook on labeling (#40)
• Enhance API to fully support all txtai functionality (#41)