DDSP Colab Notebook - Pip Install is not worked

[filipeborato]

Running the first cell, pip install is not working in the google colab. Apparently it is a problem in the tensor flow library that generates several warnings and does not finish the process.

[zwrok]

I have the same problem. Few days ago code was working with no issues.

[digwit678]

same problem here as well - cell does not work anymore since last week. Anybody knows when they will look into this because I wanted to work with this notebook ?

[zwrok]

Deleting the specific install version from ddsp==1.6.5 to just ddsp is removing error for me- but still next cells are giving a other problems. So not sure if this is a right way. Fix for next cell (upload audio) is to start it with :

from tensorflow.python.ops.numpy_ops import np_config
np_config.enable_numpy_behavior()

but on modify conditioning cell it gives error :

'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'copy'
refering to line audio_features_mod = {k: v.copy() for k, v in audio_features.items()}

[olaviinha]

🚫 UPDATE: This solution was deprecated in March 2023 after Colab upgraded to Python 3.9

---

This workaround resolves the neverending pip installation issue:

I.e. install the latest version 3.5.0 first (notebook will not work with that), then the older version. My guess is installing ddsp==3.5.0 will resolve some pip package dependency issues that will otherwise cause the infinity install on 1.6.5.

!pip install -qU ddsp==3.5.0
!pip install -qU ddsp==1.6.5 "hmmlearn<=0.2.7"

[olaviinha]

🚫 UPDATE: This solution was deprecated in April 2023. It will now keep crashing the runtime and/or behaving strangely, as warned in the installation process.

---

New workaround below. DO NOT change the order of these lines and DO NOT restart runtime despite such suggestions/buttons provided in the process of executing this snippet.

# Install hmmlearn before downgrading Python
!pip install hmmlearn

# Downgrade Python
!apt-get update -y
!apt-get install python3.8
!update-alternatives --set python3 /usr/bin/python3.8
!curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
!python get-pip.py
import sys
_ = (sys.path.append("/usr/local/lib/python3.8/dist-packages"))

# Preinstall legacy packages
!pip install numba==0.48
!pip install numpy==1.23
!pip install packaging==21.3
!pip install librosa==0.7.2

# Install Magenta & legacy DDSP
!pip install magenta
!pip install ddsp==1.9.0

[olaviinha]

🚫 UPDATE: This solution is deprecated

---

New workaround below. Note that this solution will give the same warning as the solution above WARNING: Upgrading ipython, ipykernel, tornado, prompt-toolkit or pyzmq can cause your runtime to repeatedly crash or behave in unexpected... –  only this time it can be ignored and will in fact not repeatedly crash or do other weird stuff.

Again: DO NOT change the order of these lines and DO NOT restart runtime despite such suggestions/buttons possibly provided in the process of executing this snippet.

# Install hmmlearn before downgrading Python
!pip install hmmlearn

# Downgrade Python
!apt-get update -y
!apt-get install python3.8
!update-alternatives --set python3 /usr/bin/python3.8
!curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
!python get-pip.py
import sys
_ = (sys.path.append("/usr/local/lib/python3.8/dist-packages"))

# Preinstall legacy packages
!pip install numba==0.48
!pip install numpy==1.23
!pip install packaging==21.3
!pip install librosa==0.7.2
!pip install ipython==7.34.0

# Install Magenta & legacy DDSP
!pip install magenta
!pip install ddsp==1.9.0

[olaviinha]

This workaround is only for the timbre transfer notebook. (not for autoencoder training)

In addition to "pip install not working" it resolves a latter problem regarding collections.Iterable. I figure it's good to keep all the super filthy but notebook-enabling workarounds in this thread for now, for reachability, since it all starts with the pip installation not working.

1. Prior to doing anything else, execute this:

# Install hmmlearn before downgrading Python
!pip install hmmlearn

# Downgrade Python
!apt-get update -y
!apt-get install python3.8
!update-alternatives --set python3 /usr/bin/python3.8
!curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
!python get-pip.py
import sys
_ = (sys.path.append("/usr/local/lib/python3.8/dist-packages"))

# Preinstall legacy packages
!pip install numba==0.48
!pip install numpy==1.23
!pip install packaging==21.3
!pip install librosa==0.7.2
!pip install ipython==7.34.0

# Install Magenta & legacy DDSP
!pip install magenta
!pip install ddsp==1.9.0

# Fix Iterable by hand
from google.colab files
files.view('/usr/local/lib/python3.8/dist-packages/ddsp/core.py')

2. core.py will open in the editor on the right:
   -> Replace lline 47 with return x if isinstance(x, collections.abc.Iterable) else [x]
   i.e. collections.Iterable to collections.abc.Iterable

[moedefm]

hey, thanks for the help, but unfortunately it doesnt work for me.

the core.py opens up but the line is already written correctly.

after running the next cell to load a model i get this output:
Loading dataset statistics from /content/pretrained/dataset_statistics.pkl

---

AttributeError Traceback (most recent call last)

in <cell line: 98>()
96
97 # Set up the model just to predict audio given new conditioning
---> 98 model = ddsp.training.models.Autoencoder()
99 model.restore(ckpt)
100

12 frames

/usr/local/lib/python3.10/dist-packages/gin/config.py in gin_wrapper(*args, **kwargs)
1543 # See the docstring on ConfigurableReference.__deepcopy__ above for more
1544 # details on the dark magic happening here.
-> 1545 new_kwargs = copy.deepcopy(new_kwargs)
1546
1547 # Validate args marked as REQUIRED have been bound in the Gin config.

/usr/lib/python3.10/copy.py in deepcopy(x, memo, _nil)
144 copier = _deepcopy_dispatch.get(cls)
145 if copier is not None:
--> 146 y = copier(x, memo)
147 else:
148 if issubclass(cls, type):

/usr/lib/python3.10/copy.py in _deepcopy_dict(x, memo, deepcopy)
229 memo[id(x)] = y
230 for key, value in x.items():
--> 231 y[deepcopy(key, memo)] = deepcopy(value, memo)
232 return y
233 d[dict] = _deepcopy_dict

/usr/lib/python3.10/copy.py in deepcopy(x, memo, _nil)
151 copier = getattr(x, "deepcopy", None)
152 if copier is not None:
--> 153 y = copier(memo)
154 else:
155 reductor = dispatch_table.get(cls)

/usr/local/lib/python3.10/dist-packages/gin/config.py in deepcopy(self, memo)
776 """
777 if self._evaluate:
--> 778 return self._scoped_configurable_fn()
779 return self._scoped_configurable_fn
780

/usr/local/lib/python3.10/dist-packages/gin/config.py in gin_wrapper(*args, **kwargs)
1603 scope_info = " in scope '{}'".format(scope_str) if scope_str else ''
1604 err_str = err_str.format(name, fn_or_cls, scope_info)
-> 1605 utils.augment_exception_message_and_reraise(e, err_str)
1606
1607 return gin_wrapper

/usr/local/lib/python3.10/dist-packages/gin/utils.py in augment_exception_message_and_reraise(exception, message)
39 proxy = ExceptionProxy()
40 ExceptionProxy.qualname = type(exception).qualname
---> 41 raise proxy.with_traceback(exception.traceback) from None
42
43

/usr/local/lib/python3.10/dist-packages/gin/config.py in gin_wrapper(*args, **kwargs)
1580
1581 try:
-> 1582 return fn(*new_args, **new_kwargs)
1583 except Exception as e: # pylint: disable=broad-except
1584 err_str = ''

/usr/local/lib/python3.10/dist-packages/gin/config.py in meta_call_wrapper(new_cls, *args, **kwargs)
514 if new_cls.bases == (cls,):
515 new_cls = cls
--> 516 return cls_meta.call(new_cls, *args, **kwargs)
517
518 return meta_call_wrapper

/usr/local/lib/python3.8/dist-packages/ddsp/training/preprocessing.py in init(self, time_steps, **kwargs)
62
63 def init(self, time_steps=1000, **kwargs):
---> 64 super().init(**kwargs)
65 self.time_steps = time_steps
66

/usr/local/lib/python3.8/dist-packages/ddsp/training/nn.py in init(self, input_keys, output_keys, **kwargs)
82 self.default_input_keys = []
83 self.default_input_values = []
---> 84 output_keys = output_keys or self.get_return_annotations('call')
85
86 self.input_keys = list(input_keys)

/usr/local/lib/python3.8/dist-packages/ddsp/training/nn.py in get_return_annotations(self, method)
232 """Get list of strings of return annotations of method."""
233 spec = inspect.getfullargspec(getattr(self, method))
--> 234 return core.make_iterable(spec.annotations['return'])
235
236

/usr/local/lib/python3.8/dist-packages/ddsp/core.py in make_iterable(x)
45 return [x]
46 else:
---> 47 return x if isinstance(x, collections.abc.iterable) else [x]
48
49 def to_dict(x, keys):

AttributeError: module 'collections' has no attribute 'Iterable'
In call to configurable 'F0LoudnessPreprocessor' (<class 'ddsp.training.preprocessing.F0LoudnessPreprocessor'>)

[yagizolmez]

I have the same issue.

[RamakanthRGunishetty]

---

AttributeError Traceback (most recent call last)
in <cell line: 98>()
96
97 # Set up the model just to predict audio given new conditioning
---> 98 model = ddsp.training.models.Autoencoder()
99 model.restore(ckpt)
100

12 frames
/usr/local/lib/python3.8/dist-packages/ddsp/core.py in make_iterable(x)
45 return [x]
46 else:
---> 47 return x if isinstance(x, collections.Iterable) else [x]
48
49

AttributeError: module 'collections' has no attribute 'Iterable'
In call to configurable 'F0LoudnessPreprocessor' (<class 'ddsp.training.preprocessing.F0LoudnessPreprocessor'>)

I got the same error ,how to resolve this error

[olaviinha]

Gave up on workarounds and just created modified fork that works in Colab: https://github.com/olaviinha/ddsp
At least synthesizing appears to work, will see about training later.

[HuntZhaozq]

Deleting the specific install version from ddsp==1.6.5 to just ddsp is removing error for me- but still next cells are giving a other problems. So not sure if this is a right way. Fix for next cell (upload audio) is to start it with :

from tensorflow.python.ops.numpy_ops import np_config
np_config.enable_numpy_behavior()

but on modify conditioning cell it gives error :

'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'copy' refering to line audio_features_mod = {k: v.copy() for k, v in audio_features.items()}

@zwrok Hello, I meet the same problem as yours. Have you solved it? If does, could you please share some solutions with me ? Thanks a lot!