New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Roadmap #6

Open

justheuristic opened this issue Mar 15, 2022 · 0 comments

Member

justheuristic commented Mar 15, 2022

[DONE] Version 0.3: it works decently enough in all use cases

extract from private debug-1 codebase
make base transformer general enough for ViT/SWIN/VGGTransformer integration
make into a pip-installable package

Version 0.4: it works

triton backend for sparsity (@justheuristic )
fit debug-1 into a colab-like enironment
- forward + backward, 11gb gpu, 16gb ram, full sequence length
- maybe triton backend can fix that automatically?
- chunked Attn / FFN forward (Potential optimizations #4 )
customizable checkpoints (Potential optimizations #4 )
some vision architectures (Vision transformers #2 )
basic inference for LeanGPT ( @xtinkt + @justheuristic )

Version 0.4-0.6:

memory-efficient attention kernel (@krunt + @xtinkt )
feature-rich inference (with prefices, various sampling & beam search variants)
pre-trained model zoo (convert sahaj2, calm, debug0 & 1, rubert)
demonstrable memory savings & performance comparison
efficient sota vision architectures

Finale:

If it works fine, consinder merging into HF transformers

justheuristic pinned this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment