Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MMDiT implementation and text-to-image training with rectified flows #155

Merged
merged 27 commits into from
Jul 26, 2024
Merged

MMDiT implementation and text-to-image training with rectified flows #155

merged 27 commits into from
Jul 26, 2024

Commits on Jul 25, 2024

  1. Initial transformer

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    293d30d View commit details
    Browse the repository at this point in the history

  2. Initial running composer model

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    a882d67 View commit details
    Browse the repository at this point in the history

  3. Configurable model size

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    307bc12 View commit details
    Browse the repository at this point in the history

  4. Fix calculation of input sequence length

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    e2dd53e View commit details
    Browse the repository at this point in the history

  5. Forgot downsample factor in sequence length calc

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    3317a09 View commit details
    Browse the repository at this point in the history

  6. Need affines for the layernorms

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    312c13e View commit details
    Browse the repository at this point in the history

  7. Turn off weight decay for biases, norms, and position embeddings

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    36b3432 View commit details
    Browse the repository at this point in the history

  8. Refactor and add tests

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    40b21cc View commit details
    Browse the repository at this point in the history

  9. Wrapping, pooled conditioning, flop calc fix

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    b396fa6 View commit details
    Browse the repository at this point in the history

  10. Initial working MMDiT implementation

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    36df692 View commit details
    Browse the repository at this point in the history

  11. Simplify mask logic

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    d440142 View commit details
    Browse the repository at this point in the history

  12. Rectified flows and pooled embeddings

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    de3d50e View commit details
    Browse the repository at this point in the history

  13. Prep for inference

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    2ad5fc2 View commit details
    Browse the repository at this point in the history

  14. Pooled embeddings should be zeroed after embedding for cfg

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    499e7ee View commit details
    Browse the repository at this point in the history

  15. Use shared functions to reduce error surface

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    d06580a View commit details
    Browse the repository at this point in the history

  16. Docs and a subtle timestep bug fix

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    4e21bc0 View commit details
    Browse the repository at this point in the history

  17. Docs and types for transformer

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    602c0a9 View commit details
    Browse the repository at this point in the history

  18. Refactor composer model to be separate from base transformer

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    a59ee6d View commit details
    Browse the repository at this point in the history

  19. Docs and types for composer model

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    5db34d4 View commit details
    Browse the repository at this point in the history

  20. Add dummy pretrained flag

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    f2ffcb6 View commit details
    Browse the repository at this point in the history

  21. Minor cleanup

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    b8aaa4c View commit details
    Browse the repository at this point in the history

  22. Update tests

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    40b5d9f View commit details
    Browse the repository at this point in the history

  23. Figure out max input sequence length from image size

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    f62f647 View commit details
    Browse the repository at this point in the history

  24. Equally spaced timesteps during eval

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    0df0095 View commit details
    Browse the repository at this point in the history

  25. Separate AdaLN and modulation modules

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    ab641eb View commit details
    Browse the repository at this point in the history

  26. Fix formatting

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    ce55ee0 View commit details
    Browse the repository at this point in the history

  27. Some tests are gpu only

    @corystephenson-db
    corystephenson-db committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    0f0ca90 View commit details
    Browse the repository at this point in the history