Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unknown PCM cards and ERROR not all tensors loaded from model file - expected 245, got 3 #11

Open
robotx10 opened this issue Nov 27, 2023 · 2 comments
Open

Unknown PCM cards and ERROR not all tensors loaded from model file - expected 245, got 3 #11

robotx10 opened this issue Nov 27, 2023 · 2 comments

Comments

@robotx10
Copy link

admin@ubuntu:/workspaces/isaac_ros-dev$ ros2 launch whisper_bringup bringup.launch.py n_thread:=4
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-11-27-14-33-17-107142-ubuntu-172426
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [audio_listener-1]: process started with pid [172443]
[INFO] [component_container_mt-2]: process started with pid [172445]
[component_container_mt-2] [INFO] [1701124397.745416739] [whisper_container]: Load Library: /workspaces/isaac_ros-dev/install/whisper_server/lib/libinference_component.so
[component_container_mt-2] [INFO] [1701124397.757107988] [whisper_container]: Found class: rclcpp_components::NodeFactoryTemplatewhisper::InferenceComponent
[component_container_mt-2] [INFO] [1701124397.757246584] [whisper_container]: Instantiate class: rclcpp_components::NodeFactoryTemplatewhisper::InferenceComponent
[component_container_mt-2] Path already exists: /home/admin/.cache/whisper.cpp
[component_container_mt-2] [INFO] [1701124397.790329612] [whisper.inference]: Checking whether model base.en is available...
[component_container_mt-2] [INFO] [1701124397.790815291] [whisper.inference]: Model base.en is available.
[component_container_mt-2] [INFO] [1701124397.790848477] [whisper.inference]: Initializing model base.en...
[component_container_mt-2] whisper_init_from_file_with_params_no_state: loading model from '/home/admin/.cache/whisper.cpp/ggml-base.en.bin'
[component_container_mt-2] whisper_model_load: loading model
[component_container_mt-2] whisper_model_load: n_vocab = 51864
[component_container_mt-2] whisper_model_load: n_audio_ctx = 1500
[component_container_mt-2] whisper_model_load: n_audio_state = 512
[component_container_mt-2] whisper_model_load: n_audio_head = 8
[component_container_mt-2] whisper_model_load: n_audio_layer = 6
[component_container_mt-2] whisper_model_load: n_text_ctx = 448
[component_container_mt-2] whisper_model_load: n_text_state = 512
[component_container_mt-2] whisper_model_load: n_text_head = 8
[component_container_mt-2] whisper_model_load: n_text_layer = 6
[component_container_mt-2] whisper_model_load: n_mels = 80
[component_container_mt-2] whisper_model_load: ftype = 1
[component_container_mt-2] whisper_model_load: qntvr = 0
[component_container_mt-2] whisper_model_load: type = 2 (base)
[component_container_mt-2] whisper_model_load: adding 1607 extra tokens
[component_container_mt-2] whisper_model_load: n_langs = 99
[component_container_mt-2] ggml_init_cublas: GGML_CUDA_FORCE_MMQ: no
[component_container_mt-2] ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
[component_container_mt-2] ggml_init_cublas: found 1 CUDA devices:
[component_container_mt-2] Device 0: Orin, compute capability 8.7
[audio_listener-1] ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
[audio_listener-1] ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
[audio_listener-1] ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
[audio_listener-1] ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
[audio_listener-1] ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
[audio_listener-1] ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
[audio_listener-1] ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
[audio_listener-1] ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
[audio_listener-1] ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
[audio_listener-1] ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
[audio_listener-1] ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
[audio_listener-1] ALSA lib pulse.c:242:(pulse_connect) PulseAudio: Unable to connect: Connection refused
[audio_listener-1]
[audio_listener-1] ALSA lib pulse.c:242:(pulse_connect) PulseAudio: Unable to connect: Connection refused
[audio_listener-1]
[audio_listener-1] ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card
[audio_listener-1] ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card
[component_container_mt-2] whisper_model_load: CPU buffer size = 156.67 MB
[component_container_mt-2] whisper_model_load: model size = 57.10 MB
[component_container_mt-2] whisper_model_load: ERROR not all tensors loaded from model file - expected 245, got 3
[component_container_mt-2] whisper_init_with_params_no_state: failed to load model
[component_container_mt-2] [INFO] [1701124401.647853668] [whisper.inference]: Model base.en initialized.
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/whisper/inference' in container '/whisper_container'
^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT)
[component_container_mt-2] [INFO] [1701124438.407245342] [rclcpp]: signal_handler(signum=2)
[ERROR] [audio_listener-1]: process has died [pid 172443, exit code -2, cmd '/workspaces/isaac_ros-dev/install/audio_listener/lib/audio_listener/audio_listener --ros-args'].
[audio_listener-1] Traceback (most recent call last):
[audio_listener-1] File "/workspaces/isaac_ros-dev/install/audio_listener/lib/audio_listener/audio_listener", line 33, in
[audio_listener-1] sys.exit(load_entry_point('audio-listener', 'console_scripts', 'audio_listener')())
[audio_listener-1] File "/workspaces/isaac_ros-dev/build/audio_listener/audio_listener/audio_listener.py", line 70, in main
[audio_listener-1] rclpy.spin(audio_listener)
[audio_listener-1] File "/opt/ros/humble/lib/python3.8/site-packages/rclpy/init.py", line 222, in spin
[audio_listener-1] executor.spin_once()
[audio_listener-1] File "/opt/ros/humble/lib/python3.8/site-packages/rclpy/executors.py", line 705, in spin_once
[audio_listener-1] handler, entity, node = self.wait_for_ready_callbacks(timeout_sec=timeout_sec)
[audio_listener-1] File "/opt/ros/humble/lib/python3.8/site-packages/rclpy/executors.py", line 691, in wait_for_ready_callbacks
[audio_listener-1] return next(self._cb_iter)
[audio_listener-1] File "/opt/ros/humble/lib/python3.8/site-packages/rclpy/executors.py", line 588, in _wait_for_ready_callbacks
[audio_listener-1] wait_set.wait(timeout_nsec)
[audio_listener-1] KeyboardInterrupt
[INFO] [component_container_mt-2]: process has finished cleanly [pid 172445]
@robotx10
Copy link
Author

[audio_listener-1] Traceback (most recent call last):
[audio_listener-1] File "/workspaces/isaac_ros-dev/install/audio_listener/lib/audio_listener/audio_listener", line 33, in
[audio_listener-1] sys.exit(load_entry_point('audio-listener', 'console_scripts', 'audio_listener')())
[audio_listener-1] File "/workspaces/isaac_ros-dev/build/audio_listener/audio_listener/audio_listener.py", line 73, in main
[audio_listener-1] rclpy.spin(audio_listener)
[audio_listener-1] File "/opt/ros/humble/lib/python3.8/site-packages/rclpy/init.py", line 222, in spin
[audio_listener-1] executor.spin_once()
[audio_listener-1] File "/opt/ros/humble/lib/python3.8/site-packages/rclpy/executors.py", line 713, in spin_once
[audio_listener-1] raise handler.exception()
[audio_listener-1] File "/opt/ros/humble/lib/python3.8/site-packages/rclpy/task.py", line 239, in call
[audio_listener-1] self.handler.send(None)
[audio_listener-1] File "/opt/ros/humble/lib/python3.8/site-packages/rclpy/executors.py", line 418, in handler
[audio_listener-1] await call_coroutine(entity, arg)
[audio_listener-1] File "/opt/ros/humble/lib/python3.8/site-packages/rclpy/executors.py", line 332, in execute_timer
[audio_listener-1] await await_or_execute(tmr.callback)
[audio_listener-1] File "/opt/ros/humble/lib/python3.8/site-packages/rclpy/executors.py", line 107, in await_or_execute
[audio_listener-1] return callback(*args)
[audio_listener-1] File "/workspaces/isaac_ros-dev/build/audio_listener/audio_listener/audio_listener.py", line 55, in audio_publisher_timer_callback
[audio_listener-1] audio = self.stream.read(self.frames_per_buffer_)
[audio_listener-1] File "/home/admin/.local/lib/python3.8/site-packages/pyaudio/init.py", line 570, in read
[audio_listener-1] return pa.read_stream(self._stream, num_frames,
[audio_listener-1] OSError: [Errno -9981] Input overflowed
[ERROR] [audio_listener-1]: process has died [pid 198921, exit code 1, cmd '/workspaces/isaac_ros-dev/install/audio_listener/lib/audio_listener/audio_listener --ros-args'].

managed to get the data from speaker on /audio_listener/audio but seems unreliable OSError: [Errno -9981] Input overflowed

@mhubii
Copy link
Collaborator

mhubii commented Dec 5, 2023 •
edited
Loading

hi @robotx10 , any idea why this happens? Seems to be related to pyaudio

Have you tried only running the audio listener node? E.g.

ros2 run audio_listener audio_listener

then echo or hz to verify the audio works

ros2 topic hz /audio_listener/audio

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mhubii @robotx10