Add support for authenticated calls to the Spotify Web API

[kingosticks]

Spotify recently announced the following:

On 29th May 2017 the /search /tracks /albums /artists and /users (and related) endpoints will start requiring an access token, bringing them in-line with all other Web API endpoints

We currently access the /search and /tracks endpoints so we'll soon need to use Spotify's Authorization Code Flow to continue doing so. The good news is that the infrastructure for doing this has already been setup and is used by mopidy-spotify-web. Merging mopidy-spotify-web and mopidy-spotify together (#114) makes even more sense and might be the first place to start.

We would then be able to start using the Web API for more of the features currently provided by libspotify and reduce our dependency on that deprecated library (#110). Except, of course, for streaming which is unfortunately still only possible through libspotify.

The Web API also give us access to features/improvements that are not available in libspotify, such as:

• 'Your Music' (Support for the new "Your Music" container #16).
• 'Top Lists' / Discover Weekly / Release Radar (Unhandled exception when playlist loading times out (MPD list command) #122)
• Playlist images (Add support for playlist images to get_images() #100)
• Better searching (Searching for exact matches #11)

[tomkel]

The features that stem from the proposed merge would be greatly appreciated.

[adamcik]

I started on https://github.com/mopidy/mopidy-spotify/tree/feature/oauth based on some other OAuth things I've done before. But I've not had time to fix the existing tests or add new ones. But this should be enough to survive the end-points that should soon require auth.

So if someone else has time to finish this up we could get a new release out in time for the API change. And then longer term someone could consider #114 with respect to this.

[tmb]

I think the API change happened... my search refuses to work (just happened today!)

[relthyg]

Can confirm the API change. https://api.spotify.com/v1/search?type=artist&q=Opeth returns:

{
  "error" : {
    "status" : 401,
    "message" : "No token provided"
  }
}

[stphnnnn]

Is there a plan of action to get this fixed?

[kingosticks]

Someone needs to fix up the tests for https://github.com/mopidy/mopidy-spotify/tree/feature/oauth. In the meantime you can use that branch if you want:
pip install https://github.com/mopidy/mopidy-spotify/archive/feature/oauth.zip

[blacklight]

The search is still broken also on the new OAuth branch.

Btw it's quite a shame that we had to wait for things to really break badly before supporting OAuth decently, despite Spotify provided a very very early notice about the deprecation of the old Spotify API...

[relthyg]

The search is still broken also on the new OAuth branch.

Cant confirm that. For me it works.

[blacklight]

My search attempts fail with errors such as:

May 30 16:34:27 einstein mopidy[25491]: INFO     Searching SoundCloud for 'riverside'
May 30 16:34:28 einstein mopidy[25491]: ERROR    SpotifyBackend backend caused an exception.
May 30 16:34:28 einstein mopidy[25491]: Traceback (most recent call last):
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/mopidy/core/library.py", line 19, in _backend_error_handling
May 30 16:34:28 einstein mopidy[25491]:     yield
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/mopidy/core/library.py", line 347, in search
May 30 16:34:28 einstein mopidy[25491]:     result = future.get()
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/pykka/threading.py", line 52, in get
May 30 16:34:28 einstein mopidy[25491]:     compat.reraise(*self._data['exc_info'])
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/pykka/compat.py", line 12, in reraise
May 30 16:34:28 einstein mopidy[25491]:     exec('raise tp, value, tb')
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/pykka/actor.py", line 201, in _actor_loop
May 30 16:34:28 einstein mopidy[25491]:     response = self._handle_receive(message)
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/pykka/actor.py", line 295, in _handle_receive
May 30 16:34:28 einstein mopidy[25491]:     return callee(*message['args'], **message['kwargs'])
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/mopidy_spotify/library.py", line 37, in search
May 30 16:34:28 einstein mopidy[25491]:     query, uris, exact)
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/mopidy_spotify/search.py", line 62, in search
May 30 16:34:28 einstein mopidy[25491]:     'type': ','.join(types)})
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/mopidy_spotify/web.py", line 79, in get
May 30 16:34:28 einstein mopidy[25491]:     self._refresh_token()
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/mopidy_spotify/web.py", line 56, in _refresh_token
May 30 16:34:28 einstein mopidy[25491]:     data={'grant_type': 'client_credentials'})
May 30 16:34:28 einstein mopidy[25491]:   File "/usr/lib/python2.7/site-packages/mopidy_spotify/web.py", line 41, in _request
May 30 16:34:28 einstein mopidy[25491]:     raise Error('Fetching %s failed: %s' % (url, e))
May 30 16:34:28 einstein mopidy[25491]: Error: Fetching https://auth.mopidy.com/spotify/token failed: 401 Client Error: UNAUTHORIZED for url: https://auth.mopidy.com/spotify/token

[kingosticks]

Did you visit https://www.mopidy.com/authenticate/#spotify to get your client_id and client_secret ? Looks like you did not.

[spotify]
client_id = ... client_id value you got from mopidy.com ...
client_secret = ... client_secret value you got from mopidy.com ...

EDIT: I've updated the branch README to show this.

[blacklight]

I did, but the situation with the client_id seems to be quite messy now.

You are mentioning that the client_id configuration should be in the [spotify] section, while the link you posted mentions the [spotify_web] section.

The fact is that I had two different client_ids in those sections - whatever was there before in [spotify_web], and the newly generated one for [spotify], and that configuration didn't work. Using the same for both fixes the search.

So, right now we apparently ought to use the same configuration in two modules which are supposed to be distinct.

[kingosticks]

Please, feel free to rectify the situation. The code should all be available on here (and at https://github.com/adamcik/oauthclientbridge)

[adamcik]

Possible gotchas and improvements:

• old oauth authorizations might have too limited a scope, but this is likely a non issue for this stage.
• we should probably check for id/secret from spotify_web, though that would mean making the new ones optional?
• tests can probably be fixed by disabling token fetching in test and adding tests for the client

[timdp]

Confirming that manually upgrading and configuring OAuth credentials fixes this.

I'd installed it using apt (on Raspbian), so rather than trying to upgrade with pip, I replaced the contents of /usr/lib/python2.7/dist-packages/mopidy_spotify. Not pretty, but it fixes it for now.

[kingosticks]

I updated the tests at #143

[kingosticks]

we should probably check for id/secret from spotify_web, though that would mean making the new ones optional?

What if we keep them required and just check that they match those defined for spotify_web (if it's enabled)?

[jodal]

Fixed by #143. Mopidy-Spotify 3.1.0 has been released to PyPI, apt.mopidy.com and Homebrew.

[trygveaa]

@grighi: You're mixing up Mopidy and Mopidy-Spotify. It is Mopidy-Spotify that has been released in version 3.1, and that version is published on GitHub. The Arch Linux Community repo doesn't have Mopidy-Spotify, only Mopidy (which it has in version 2.1 which is the most recent). AUR has Mopidy-Spotify in version 3.1.

[mdundek]

I am running mopidy 2.1 & mopidy-spotify 3.1 on a raspberry pi (raspbian stretch, not jessy), and I can not get around the 401 exception when searching using http api. It used to work on a older Jessy installation I had, but I have no other choice than to move to Stretch now. I started from scratch like 3 times now, always the same results. Strange thing is that nobody else seems to complain about this configuration, so I really start to wonder if I didn't miss something big here...
In case you wonder, I did configure the spotify account and keys in my config file, the mopidy logs tell me it's all good, but when I search, I get the following error:

Error: Fetching https://auth.mopidy.com/spotify/token failed: 401 Client Error: UNAUTHORIZED for url: https://auth.mopidy.com/spotify/token

Can anyone else confirm if they got the search to work on Stretch?

[paufabregat]

@mdundek I got it working on Stretch. I didn't do anything in particular beyond the usual installation and configuration, make sure you set the proper client_id and client_secret retrieved from here https://www.mopidy.com/authenticate/#spotify

Running mopidy 2.1 & mopidy-spotify 3.1 as well.