Spirc: Replace Mecury with Dealer #1356

photovoltex · 2024-09-29T15:07:37Z

Initial PR Description

As suggested from @roderickvd a draft PR so i can have some community feedback and give now and then updates on the progress.

The replacement is in a working state, so the very basic do work again. There is still a lot of cleanup and re-implementing to do.

But so far what is working:

transferring the playback so that we are the player
- funnily enough, transferring back isn't working yet
play, pause, resume, prev, next, seek
just general communication of our state to other clients

Previously Known Problems:

on the initial connection librespot doesn't appear, but if for example the volume is slightly updated, we should appear

this might be a web-player specific problem, in the client the dealer just pops up
- this seems to be to some degree fine? if the devices are reopend it seems to be always there, it just isn't dynamically updated.
if the session or spirc/dealer dies, we lose our complete ConnectState
- it would be nice if that is preserved so we could start from that again
sometimes other client's forget that we are the current player
- ~~might be that that is already resolved, but i think i still saw that behavior not so long ago~~
- update: seems to be resolved, was probably just an early development problem

What i still have to do:

I tried to keep my code clean, but if there are points that i could improve and such, i would like to here them^^

Currently Known Issues:

podcasts/episodes don't work
- see Spirc: Replace Mecury with Dealer #1356 (comment)
starting a track out of "recent search" from mobile and then trying to switch back to mobile
- see Spirc: Replace Mecury with Dealer #1356 (comment)
- seems to get into a weird state where the mobile app is completely desynced
on mobile it can occur that librespot is seen as still playing with the current account, even tho it isn't playing
shufflling a context with multiple pages does only shuffle the first page (because initially only that page is loaded)
- this is a problem for example, when shuffling an artist context

Fixes #19

the dealer does support this via the field repeating_track and set_options command

Fixes #434

a context is now resolved completely

Fixes #442

a context is now resolved completely

Fixes #701

the dealer does support this action via the command set_queue

Fixes #765

resolve context does provide the top 10 tracks and all albums in order from newest to oldest

Fixes #861

the next tracks are completely reworked

Fixes #1205

thx @PocketMiner82 for helping test :)

Fixes #1353

I encountered this issue during development, but after fixing everything around the uid's it seems to be fully functional now

Thx again to all people testing the branch, helps a lot :D

photovoltex · 2024-09-29T15:19:07Z

oh the branch might not compile because there are still some todo!() here and there which result in unreachable statements that clippy, understandably, doesn't like.

roderickvd · 2024-09-30T19:47:05Z

Super job! I know you're not there yet, so let's all contribute here. Stuff like this almost makes me sign up for a Spotify plan again 😉

I'm gonna have a more thorough look later. Wanted to ping back on your comment over at #1349:

The hm://pusher/v1/connections/ message is send after connecting to set up some connect stuff initially. Can be safely ignored as far as i know. It just falls under one of the many unknown subscriptions category that are logged.

If you look at librespot-java, you'll see that it listens to this message to extract the Spotify-Connection-Id from the headers:

https://github.com/librespot-org/librespot-java/blob/d0ff31b4f476590c9c17401aac01a6762e4ba908/player/src/main/java/xyz/gianlu/librespot/player/state/DeviceStateHandler.java#L162

I think if you follow the code flow, updating the connection ID then triggers the ready event in the deviceStateListener, which in turn gets the volume. And that seems to gel with you saying that:

on the initial connection librespot doesn't appear, but if for example the volume is slightly updated, we should appear

- impl shuffle again - extracted fill up of next tracks in own method - moved queue revision update into next track fill up - removed unused method `set_playing_track_index` - added option to specify index when resetting the playback context - reshuffle after repeat context

PocketMiner82 · 2024-12-04T08:30:03Z

I additionally improved the context handling again.
And for myself, the issue @PocketMiner82 described around the transferring of the recent search seems to be resolved by that :D

@photovoltex yes, it is indeed working now :)

roderickvd · 2024-12-05T13:14:54Z

Awesome so let's get it in dev. Would you consider squashing it?

PocketMiner82 · 2024-12-06T09:51:31Z

Hmmm... I think I found a new issue, happened to me 3 times today (but for some reason never before): I was listening to the liked songs playlists for some time and suddenly the music stops playing in the middle of a song. the following errors got logged:

Dec 06 10:39:52 rpi-3a librespot[1607]: [2024-12-06T09:39:52Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:00 rpi-3a librespot[1607]: [2024-12-06T09:40:00Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:08 rpi-3a librespot[1607]: [2024-12-06T09:40:08Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:16 rpi-3a librespot[1607]: [2024-12-06T09:40:16Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:24 rpi-3a librespot[1607]: [2024-12-06T09:40:24Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:32 rpi-3a librespot[1607]: [2024-12-06T09:40:32Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:40 rpi-3a librespot[1607]: [2024-12-06T09:40:40Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:48 rpi-3a librespot[1607]: [2024-12-06T09:40:48Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:56 rpi-3a librespot[1607]: [2024-12-06T09:40:56Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:41:04 rpi-3a librespot[1607]: [2024-12-06T09:41:04Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")

Note that the wifi of this device (Raspberry Pi 3 A+) is a bit unstable for me, sometimes it doesn't respond for ~15 seconds... Maybe that's why the wait timeout exceeded. I thought I'll report it regardless, maybe some timeout can be increased 🤷

photovoltex · 2024-12-06T12:36:14Z

Awesome so let's get it in dev. Would you consider squashing it?

Sure thing. But did anyone look over the code or do we just merge it? I could understand why, the changes are a bit too drastic/large to review. But just wanted to hear what the stance in regards to that is^^

roderickvd · 2024-12-06T12:41:20Z

In an ideal world I'd like to run through the code, but I don't have a lot of time for it in the coming weeks. So I thought that it's better to get it in and we can still place code comments on GitHub after the merge.

photovoltex · 2024-12-06T12:54:27Z

Sounds reasonable. Feel free to merge whenever fits best :)

photovoltex · 2024-12-06T12:57:30Z

Hmmm... I think I found a new issue, happened to me 3 times today (but for some reason never before): I was listening to the liked songs playlists for some time and suddenly the music stops playing in the middle of a song. the following errors got logged:

Dec 06 10:39:52 rpi-3a librespot[1607]: [2024-12-06T09:39:52Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:00 rpi-3a librespot[1607]: [2024-12-06T09:40:00Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:08 rpi-3a librespot[1607]: [2024-12-06T09:40:08Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:16 rpi-3a librespot[1607]: [2024-12-06T09:40:16Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:24 rpi-3a librespot[1607]: [2024-12-06T09:40:24Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:32 rpi-3a librespot[1607]: [2024-12-06T09:40:32Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:40 rpi-3a librespot[1607]: [2024-12-06T09:40:40Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:48 rpi-3a librespot[1607]: [2024-12-06T09:40:48Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:56 rpi-3a librespot[1607]: [2024-12-06T09:40:56Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:41:04 rpi-3a librespot[1607]: [2024-12-06T09:41:04Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")

Note that the wifi of this device (Raspberry Pi 3 A+) is a bit unstable for me, sometimes it doesn't respond for ~15 seconds... Maybe that's why the wait timeout exceeded. I thought I'll report it regardless, maybe some timeout can be increased 🤷

That should probably be unrelated to these changes. But if they appear again we can open an issue^^

YutongGu · 2024-12-06T22:57:35Z

Hmmm... I think I found a new issue, happened to me 3 times today (but for some reason never before): I was listening to the liked songs playlists for some time and suddenly the music stops playing in the middle of a song. the following errors got logged...

I can report that this issue occurred for me as well:

[2024-12-06T21:34:29Z INFO  librespot_playback::player] Loading <Guy For That (Feat. Luke Combs)> with Spotify URI <spotify:track:6StwwqB84sJeLr7tZDTxEX>
[2024-12-06T21:34:32Z ERROR librespot_core::audio_key] Audio key response timeout
Rate limited 1 logs
[2024-12-06T21:34:37Z ERROR librespot_core::session] Connection to server closed.
[2024-12-06T21:35:01Z WARN  librespot_core::dealer] Websocket peer does not respond.
[2024-12-06T21:35:06Z ERROR librespot_playback::player] Unable to read audio file: Symphonia Decoder Error: Deadline expired before operation could complete { wait timeout exceeded }
[2024-12-06T21:35:08Z WARN  librespot] Spirc shut down unexpectedly
[2024-12-06T21:35:08Z ERROR librespot_core::session] Transport endpoint is not connected (os error 107)
[2024-12-06T21:35:18Z INFO  librespot_core::session] Connecting to AP "ap-guc3.spotify.com:4070"
[2024-12-06T21:35:23Z INFO  librespot_core::session] Authenticated as '1219828745' !
Rate limited 1 logs
[2024-12-06T21:35:31Z INFO  librespot_core::spclient] Resolved "guc3-spclient.spotify.com:443" as spclient access point
[2024-12-06T21:35:39Z INFO  librespot_connect::spirc] active device is <> with session <0d5cfc65538b301a627f3c9d7dce54b2>
[2024-12-06T21:38:11Z WARN  librespot_connect::state::context] switching to active context had issues: Invalid state { context is not available. type: Default }
---- 2024-12-06 21:38:14.953302: Now starting Spotify ----

photovoltex · 2024-12-07T08:48:46Z

Hui that's a lot of errors and warning in one spot. Seems all be related to sort of timeout or unresponsive network connection. Can both of you maybe try to reproduce these issues on the dev branch?

I will look into it if there might be some changes that unintentionally affect the player. But from the errors in the log it sounds more like network issues.

photovoltex · 2024-12-07T19:44:04Z

I did also find a weird behavior from the context-resolver... if you request a playlist that contains tracks and episodes, only the tracks will be resolved. The response seems to not contain any info about the episodes... I will probably not solve it anymore in context of this PR, so might be moved out into a new issue afterwards.

photovoltex · 2024-12-08T13:33:42Z

There is also an oversight with the current impl in relation to shuffle and a context with multiple pages. For example playing an artist with shuffle doesn't work yet as one would expect. That probably needs a revisit at some point in the future. Adjusting the current impl would probably add another 100+ lines to the current PR, which i would like to prevent for now^^

photovoltex · 2024-12-08T21:36:03Z

Looked again at this comment (#1356 (comment)), and it might be a better solution in the end, when we play a shuffled context we usually don't need the default context. I had hopped we could just keep refs to the original tracks in a vec, but that collides with every state modification you still want to do.

That might be a nice next task to do after this initial state is merged and we iterate on improving performance and stability :D

photovoltex · 2024-12-10T19:03:07Z

Continuing development here (https://github.com/photovoltex/librespot/tree/improve-dealer) so that this PR is only the basic integration (with some new features provided by the dealer)

roderickvd · 2024-12-10T19:31:21Z

Drumroll for merging, coming up in 3... 2....

photovoltex · 2024-12-10T20:13:31Z

Well, and now we wait for issues that we didn't spot during testing/development

photovoltex added 22 commits September 20, 2024 19:17

dealer wrapper for eas of use

736faaf

improve sending protobuf requests

1073081

replace connect config with connect_state config

dfdfbd5

start integrating dealer into spirc

2ecb8f0

payload handling, gzip support

c066c29

put connect state consistent

f8f350c

formatting

23107e7

request payload handling, gzip support

68728fe

expose dealer::protocol, move request in own file

e450428

integrate handle of connect-state commands

618473b

spirc: remove ident field

db8f6ee

transfer playing state better

c74606d

spirc: remove remote_update stream

f51d2f9

spirc: replace command sender with connect state update

b0f8fc4

spirc: remove device state and remaining unused methods

2b879f2

spirc: remove mercury sender

d487a1e

add repeat track state

4ca0e0c

ConnectState: add methods to replace state in spirc

5959fb8

spirc: move context into connect_state, update load and next

ae738dd

spirc: remove state, adjust remaining methods

a038e91

spirc: handle more dealer request commands

9684cfa

revert rustfmt.toml

959993a

photovoltex force-pushed the integrate-dealer branch from 1718360 to 959993a Compare September 29, 2024 15:11

photovoltex added 5 commits October 5, 2024 15:47

spirc: handle device became inactive

4cdd30e

dealer: adjust payload handling

a51c83f

spirc: better set volume handling

9b80245

dealer: box PlayCommand (clippy warning)

7dd5159

connect: become inactive when no cluster is reported

2aa616f

photovoltex added 2 commits December 6, 2024 13:49

update CHANGELOG.md

f9f7fc9

core/playback: preemptively fix clippy warnings

be81f2a

kingosticks mentioned this pull request Dec 6, 2024

Event Handling for AutoPlay not functioning #1411

Open

photovoltex added 3 commits December 7, 2024 19:01

connect: minor adjustment to session changed

a0c416f

connect: change return type changing active context

f258f75

connect: handle unavailable contexts

1760f16

connect: fix previous restrictions blocking load with shuffle

e4528d8

photovoltex added 2 commits December 8, 2024 14:48

connect: update comments and logging

cdbbfe0

core/connect: reduce some more duplicate code

54a5cd2

more docs around the dealer

521ea74

roderickvd merged commit 5839b36 into librespot-org:dev Dec 10, 2024
13 checks passed

photovoltex deleted the integrate-dealer branch December 10, 2024 20:14

photovoltex mentioned this pull request Dec 11, 2024

Dealer: Rework context retrieval #1414

Open

antoinecellerier mentioned this pull request Dec 14, 2024

Playlist playback stopped after a few hours with a broken pipe (os error 32) error / non recoverable websocket/TLS error (using Dealer) #1419

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Spirc: Replace Mecury with Dealer #1356

Spirc: Replace Mecury with Dealer #1356

photovoltex commented Sep 29, 2024 •

edited

Loading

photovoltex commented Sep 29, 2024

roderickvd commented Sep 30, 2024

PocketMiner82 commented Dec 4, 2024 •

edited

Loading

roderickvd commented Dec 5, 2024

PocketMiner82 commented Dec 6, 2024 •

edited

Loading

photovoltex commented Dec 6, 2024

roderickvd commented Dec 6, 2024

photovoltex commented Dec 6, 2024

photovoltex commented Dec 6, 2024 •

edited

Loading

YutongGu commented Dec 6, 2024

photovoltex commented Dec 7, 2024

photovoltex commented Dec 7, 2024

photovoltex commented Dec 8, 2024

photovoltex commented Dec 8, 2024

photovoltex commented Dec 10, 2024

roderickvd commented Dec 10, 2024

photovoltex commented Dec 10, 2024 •

edited

Loading

Spirc: Replace Mecury with Dealer #1356

Spirc: Replace Mecury with Dealer #1356

Conversation

photovoltex commented Sep 29, 2024 • edited Loading

Currently Known Issues:

photovoltex commented Sep 29, 2024

roderickvd commented Sep 30, 2024

PocketMiner82 commented Dec 4, 2024 • edited Loading

roderickvd commented Dec 5, 2024

PocketMiner82 commented Dec 6, 2024 • edited Loading

photovoltex commented Dec 6, 2024

roderickvd commented Dec 6, 2024

photovoltex commented Dec 6, 2024

photovoltex commented Dec 6, 2024 • edited Loading

YutongGu commented Dec 6, 2024

photovoltex commented Dec 7, 2024

photovoltex commented Dec 7, 2024

photovoltex commented Dec 8, 2024

photovoltex commented Dec 8, 2024

photovoltex commented Dec 10, 2024

roderickvd commented Dec 10, 2024

photovoltex commented Dec 10, 2024 • edited Loading

photovoltex commented Sep 29, 2024 •

edited

Loading

PocketMiner82 commented Dec 4, 2024 •

edited

Loading

PocketMiner82 commented Dec 6, 2024 •

edited

Loading

photovoltex commented Dec 6, 2024 •

edited

Loading

photovoltex commented Dec 10, 2024 •

edited

Loading