Problem loading new stereo music gen models (mac-os-fix)

[jasalt]

commit 882222d branch mac-os-fix

I'm having a problem loading a custom model, new stereo one. Largely new to Torch and Gradio so I think I'm doing it wrong..

I download the state_dict.bin from https://huggingface.co/facebook/musicgen-stereo-melody/tree/main and save it as musicgen-stereo-melody.pt in models/.
Gradio app finds file and I set following configuration for model in UI:

But when generating it Gradio gives error as something seems to be in wrong format:

Loading model GrandaddyShmax/musicgen-custom
RuntimeError: Error(s) in loading state_dict for LMModel:
/Users/user/.local/share/virtualenvs/audiocraft_plus-v8WMZNJZ/lib/python3.11/site-packages/torch/nn/utils/weight_norm.py:30: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
  warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
Traceback (most recent call last):
  File "/Users/user/.local/share/virtualenvs/audiocraft_plus-v8WMZNJZ/lib/python3.11/site-packages/gradio/queueing.py", line 407, in call_prediction
    output = await route_utils.call_process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.local/share/virtualenvs/audiocraft_plus-v8WMZNJZ/lib/python3.11/site-packages/gradio/route_utils.py", line 226, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.local/share/virtualenvs/audiocraft_plus-v8WMZNJZ/lib/python3.11/site-packages/gradio/blocks.py", line 1550, in process_api
    result = await self.call_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.local/share/virtualenvs/audiocraft_plus-v8WMZNJZ/lib/python3.11/site-packages/gradio/blocks.py", line 1185, in call_function
    prediction = await anyio.to_thread.run_sync(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.local/share/virtualenvs/audiocraft_plus-v8WMZNJZ/lib/python3.11/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.local/share/virtualenvs/audiocraft_plus-v8WMZNJZ/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/Users/user/.local/share/virtualenvs/audiocraft_plus-v8WMZNJZ/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.local/share/virtualenvs/audiocraft_plus-v8WMZNJZ/lib/python3.11/site-packages/gradio/utils.py", line 661, in wrapper
    response = f(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^
  File "/Users/user/dev/audio-generation/audiocraft_plus/app.py", line 738, in predict_full
    load_model(model, custom_model, base_model, gen_type)
  File "/Users/user/dev/audio-generation/audiocraft_plus/app.py", line 154, in load_model
    MODEL.lm.load_state_dict(torch.load(file_path))
  File "/Users/user/.local/share/virtualenvs/audiocraft_plus-v8WMZNJZ/lib/python3.11/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LMModel:
	Missing key(s) in state_dict: "condition_provider.conditioners.self_wav.output_proj.weight", "condition_provider.conditioners.self_wav.output_proj.bias", "condition_provider.conditioners.self_wav.chroma.spec.window", "condition_provider.conditioners.description.output_proj.weight", "condition_provider.conditioners.description.output_proj.bias", "emb.0.weight", "emb.1.weight", "emb.2.weight", "emb.3.weight", "transformer.layers.0.self_attn.in_proj_weight", "transformer.layers.0.self_attn.out_proj.weight", "transformer.layers.0.linear1.weight", "transformer.layers.0.linear2.weight"

...

nsformer.layers.46.norm2.weight", "transformer.layers.46.norm2.bias", "transformer.layers.47.self_attn.in_proj_weight", "transformer.layers.47.self_attn.out_proj.weight", "transformer.layers.47.linear1.weight", "transformer.layers.47.linear2.weight", "transformer.layers.47.norm1.weight", "transformer.layers.47.norm1.bias", "transformer.layers.47.norm2.weight", "transformer.layers.47.norm2.bias", "out_norm.weight", "out_norm.bias", "linears.0.weight", "linears.1.weight", "linears.2.weight", "linears.3.weight".
	Unexpected key(s) in state_dict: "best_state", "xp.cfg", "version", "exported".

What would be the correct steps to get this working?

I assume I messed up cause I don't have the compression_state_dict.bin used but I couldn't figure out how to put that into models folder with .pt file extension that gradio app looks for.

Apple M1 Pro, 16GB model.

[jasalt]

Found some custom model loading documentation through musicgen discord https://discord.gg/WACPtUqH.

https://docs.google.com/document/d/1Sjx-zk5KkpHX7HIRLb6tHNacGks-XJ_cdvQTyUoBNkE/edit

Suggests both compression_state_dict.bin and state_dict.bin to reside in a subdir of models/ dir. But not working yet, not sure if it is up to date..

Trying hacky workaround to replace existing symlinks in ~/.cache/huggingface/hub/models--GrandaddyShmax--musicgen-melody/snapshots/6dd69e2b1ed71eb55301d705c40e4fbed7af4b86 to point to stereo model .bin files instead of original included models, ending up with _pickle.UnpicklingError: unpickling stack underflow..

[NeuroLord]

Could you please update the server link in discord or give working instructions on how to make the stereo model work in this GUI?

Нашел документацию по загрузке пользовательской модели через musicgen discord https://discord.gg/WACPtUqH.

https://docs.google.com/document/d/1Sjx-zk5KkpHX7HIRLb6tHNacGks-XJ_cdvQTyUoBNkE/edit

Предлагает разместить compression_state_dict.bin и state_dict.bin в подкаталоге каталога models/. Но пока не работает, не уверен, что он актуален..

Попытка хакерского обходного пути для замены существующих символических ссылок в ~/.cache/huggingface/hub/models--GrandaddyShmax--musicgen-melody/snapshots/6dd69e2b1ed71eb55301d705c40e4fbed7af4b86, чтобы они указывали на файлы стереомодели .bin вместо исходных включенных моделей, в результате чего получается _pickle.UnpicklingError: unpickling stack underflow..