Skip to content

Falcon, Code Llama, ViTDet, DINO v2, VITS
Compare
Choose a tag to compare
@LysandreJik LysandreJik released this 06 Sep 21:14
· 3732 commits to main since this release
v4.33.1
fa6107c
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Falcon

Falcon is a class of causal decoder-only models built by TII. The largest Falcon checkpoints have been trained on >=1T tokens of text, with a particular emphasis on the RefinedWeb corpus. They are made available under the Apache 2.0 license.

Falcon’s architecture is modern and optimized for inference, with multi-query attention and support for efficient attention variants like FlashAttention. Both ‘base’ models trained only as causal language models as well as ‘instruct’ models that have received further fine-tuning are available.

Code Llama

Code Llama, is a family of large language models for code based on Llama 2, providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks.

ViTDet

ViTDet reuses the ViT model architecture, adapted to object detection.

DINO v2

DINO v2 is the next iteration of the DINO model. It is added as a backbone class, allowing it to be re-used in downstream models.

VITS

VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) is an end-to-end speech synthesis model that predicts a speech waveform conditional on an input text sequence. It is a conditional variational autoencoder (VAE) comprised of a posterior encoder, decoder, and conditional prior.

Breaking changes:

  • 🚨🚨🚨 [Refactor] Move third-party related utility files into integrations/ folder 🚨🚨🚨 by @younesbelkada in #25599

Moves all third party libs (outside HF ecosystem) related utility files inside integrations/ instead of having them in transformers directly.

In order to get the previous usage you should be changing your call to the following:

- from transformers.deepspeed import HfDeepSpeedConfig
+ from transformers.integrations import HfDeepSpeedConfig

Bugfixes and improvements

Significant community contributions

The following contributors have made significant changes to the library over the last release:

  • @nablabits
    • Add type hints for several pytorch models (batch-3) (#25705)
    • Add type hints for several pytorch models (batch-2) (#25557)
    • Add type hints for pytorch models (final batch) (#25750)
    • Add type hints for several pytorch models (batch-4) (#25749)
    • Add type hints for tf models batch 1 (#25853)
    • Add type hints for tf models final batch (#25883)
  • @Lorenzobattistela
    • fixing name position_embeddings to object_queries (#24652)
  • @hollance

Contributors

tmm1, jpizarrom, and 51 other contributors
Assets 2
Loading