This is an auto-regressive implementation (char-based approach) which has its benefits but also drawbacks However, this is the most interesting, flexible, and versatile implementation that allows to try many different things
For SOTA, non-auto-regressive, token-based implementation see Optimus VIRTUOSO: Relative Global Attention Edition below
This is basically an improved reproduction of OpenAI's MuseNet. With large enough dataset, you should get comparable(or better) results
And no, you do not really need Sparse Attention/Transformers here unless you want to train on gigabytes of data
https://soundcloud.com/aleksandr-sigalov-61/sets/exclusive-preview-optimus-virtuoso
-
Multi-instrumental MIDI implementation
-
Human-readable TXT Encoding with only 3 (three) to 5 (five) UTF-8 chars per any MIDI note (5 x 4bytes/char == 20 bytes max)
-
MuseNet-like MIDI events representation
-
Composition-level MIDI events representation
-
Single-epoch/one-shot training/learning
-
GPT3 tweaks/improvements
-
Relative Global Attention (RPR)
-
Much much more...
Q) What Optimus-VIRTUOSO can do for me?
A) Optimus-VIRTUOSO can virtuosly play you any MIDI music you will train it upon
Q) What Optimus-VIRTUOSO can't do for me?
A) Optimus-VIRTUOSO can't compose fully original music for you because it is a regular Music AI (not AGI) implementation
Q) Is there any way to compose original music with Optimus-VIRTUOSO?
A) Yes! Try Optimus-VIRTUOSO Composer! It can compose compound music for you, similar to how MuseNet does it. It is not going to be fully original but it is not going to be a complete plagiarism either :)
Q) What are the possible useful practical applications for this technology?
A) Great question! Here are the major ones in order of importance and ability:
- Music Performance: Think talented AI Music Protege or an AI Music partner for a musician or a composer
- Music Composition: Think of it as an AI Music collaborator for musicians and composers
- Music Classification: This is most profitable and useful application at the moment, but this would be a rather simple and crude use of such great tech
- Music Exploration: This is also very interesting application, especially if it would be somehow tied with the #1 and #2 above
- Many more other incredible uses, so definitely give the Optimus VIRTUOSO a try :)
@inproceedings{lev2021optimusvirtuoso,
title = {Optimus-VIRTUOSO},
author = {Aleksandr Lev},
booktitle = {GitHub},
year = {2021},
}