REST API bug fix: URL-encode Unicode chars in HTTP headers

fixes https://console.cloud.google.com/errors/CLmUnOPr05b83gE

README.md

@@ -309,27 +309,29 @@ Changelog
 * Add Python 3.8 support, drop 3.3 and 3.4. Python 3.5 is now the minimum required version.
 * Add [Pixelfed](https://pixelfed.org/)! Heavily based on Mastodon.
 * Standardize Instagram's and Facebook's scraping into new common `scraped_to_activities()`, `scraped_to_activity()`, and `merge_scraped_reactions()` methods.
+* REST API:
+  * Bug fix: URL-encode Unicode characters in `Link` HTTP headers (eg `rel=self`, `rel=header`).
+* Facebook:
+  * Scraping now uses [mbasic.facebook.com](https://mbasic.facebook.com/) instead of [m.facebook.com](https://m.facebook.com/).
 * Flickr:
   * Add support for adding tags to existing photos ([bridgy#857](https://github.com/snarfed/bridgy/issues/857)).
-* JSON Feed:
-  * Gracefully handle when `content_html` and `content_text` are [incorrectly](https://jsonfeed.org/version/1#items) lists instead of strings.
+* GitHub:
+  * Handle [HTTP 451 Unavailable for Legal Reasons](https://en.wikipedia.org/wiki/HTTP_451) responses ([eg for DMCA takedowns](https://developer.github.com/changes/2016-03-17-the-451-status-code-is-now-supported/)) gracefully.
 * HTML/microformats2:
   * Add `aria-hidden="true"` to empty links ([bridgy#947](https://github.com/snarfed/bridgy/issues/947)).
   * Bug fix: escape `&`, `<`, and `>` characters in bare mf2 `content` properties ([aaronpk/XRay#102](https://github.com/aaronpk/XRay/issues/102)).
-* GitHub:
-  * Handle [HTTP 451 Unavailable for Legal Reasons](https://en.wikipedia.org/wiki/HTTP_451) responses ([eg for DMCA takedowns](https://developer.github.com/changes/2016-03-17-the-451-status-code-is-now-supported/)) gracefully.
+* JSON Feed:
+  * Gracefully handle when `content_html` and `content_text` are [incorrectly](https://jsonfeed.org/version/1#items) lists instead of strings.
 * Instagram:
   * Include threaded (ie nested) comments in scraping ([bridgy#958](https://github.com/snarfed/bridgy/issues/958)).
-* Twitter:
-  * Bug fix: URL-encode list names in API calls.
-  * Bug fix: propagate alt text into AS1 `photo.displayName` so that it gets all the way into microformats2 JSON and HTML ([#183](https://github.com/snarfed/granary/issues/183)).
 * Mastodon:
   * Bug fix for alt text with image attachments ([bridgy#975](https://github.com/snarfed/bridgy/issues/975)).
   * Omit empty `limit` param [for compatibility with Pleroma](https://git.pleroma.social/pleroma/pleroma/-/issues/2198) ([bridgy#977](https://github.com/snarfed/bridgy/issues/977)).
 * Meetup:
   * `create()`: handle API errors and return the error message in the `CreationResult` ([bridgy#921](https://github.com/snarfed/bridgy/issues/921)).
-* Facebook:
-  * Scraping now uses [mbasic.facebook.com](https://mbasic.facebook.com/) instead of [m.facebook.com](https://m.facebook.com/).
+* Twitter:
+  * Bug fix: URL-encode list names in API calls.
+  * Bug fix: propagate alt text into AS1 `photo.displayName` so that it gets all the way into microformats2 JSON and HTML ([#183](https://github.com/snarfed/granary/issues/183)).
 
 ### 3.0 - 2020-04-08
 

api.py

@@ -255,9 +255,10 @@ def write_response(self, response, actor=None, url=None, title=None,
           title=title,
           rels={'hub': hub} if hub else None,
           reader=(reader == 'true')))
-        self.response.headers.add('Link', str('<%s>; rel="self"' % self.request.url))
+        self.response.headers.add('Link', '<%s>; rel="self"' %
+                                  util.quote_path(self.request.url))
         if hub:
-          self.response.headers.add('Link', str('<%s>; rel="hub"' % hub))
+          self.response.headers.add('Link', '<%s>; rel="hub"' % util.quote_path(hub))
       elif format == 'rss':
         if not title:
           title = 'Feed for %s' % url

test_app.py

@@ -676,6 +676,17 @@ def test_hub(self):
     self.assertIn('<http://a/hub>; rel="hub"', headers)
     self.assertIn('<%s>; rel="self"' % self_url, headers)
 
+  def test_encode_urls_in_link_headers(self):
+    self.expect_requests_get('http://my/as1', AS1)
+    self.mox.ReplayAll()
+
+    url = '/url?url=http://my/as1&input=as1&output=atom&hub=http://a/%E2%98%95'
+    resp = app.application.get_response(url)
+    self.assertCountEqual(
+      ('<http://a/%E2%98%95>; rel="hub"',
+       '<http://localhost/url?url=http://my/as1&input=as1&output=atom&hub=http://a/%E2%98%95>; rel="self"'),
+      resp.headers.getall('Link'))
+
   def test_bad_mf2_json_input_400s(self):
     """If a user sends JSON Feed input, but claims it's mf2 JSON, return 400.