Code

Backoff failing RSS feeds#890

Code

Merged

Half-Shot merged 35 commits intomainmatrix-org/matrix-hookshot:mainfrom hs/add-sensible-rss-backoffmatrix-org/matrix-hookshot:hs/add-sensible-rss-backoffCopy head branch name to clipboardFeb 20, 2024

+263-67

Commits

Commits on Feb 6, 2024

• Backoff RSS requests if a url repeatedly fails.

  Half-Shot

  committed
  e78d920

  Copy full SHA for e78d920

• Increase max backoff time to a day

  Half-Shot

  committed
  f5568e2

  Copy full SHA for f5568e2

• Add backoff for failing feeds.

  Half-Shot

  committed
  2ee5012

  Copy full SHA for 2ee5012

• Remove unused finally

  Half-Shot

  committed
  b7cd4fe

  Copy full SHA for b7cd4fe

• Add this.feedLastBackoff

  Half-Shot

  committed
  18429ce

  Copy full SHA for 18429ce

• Rewrite in rust.

  Half-Shot

  committed
  80d4b9b

  Copy full SHA for 80d4b9b

• linting

  Half-Shot

  committed
  eff30b8

  Copy full SHA for eff30b8

• pop

  Half-Shot

  committed
  517d044

  Copy full SHA for 517d044

• Optimise backoff function further

  Half-Shot

  committed
  ea7af88

  Copy full SHA for ea7af88

• Drop only!

  Half-Shot

  committed
  965c30b

  Copy full SHA for 965c30b

• fix test

  Half-Shot

  committed
  e3d085e

  Copy full SHA for e3d085e

• lint

  Half-Shot

  committed
  4274eb4

  Copy full SHA for 4274eb4

• lint further

  Half-Shot

  committed
  64ab808

  Copy full SHA for 64ab808

• Better comments

  Half-Shot

  committed
  9232213

  Copy full SHA for 9232213

Commits on Feb 7, 2024

• Fix urls calculation

  Half-Shot

  committed
  d0846b1

  Copy full SHA for d0846b1

• Merge remote-tracking branch 'origin/main' into hs/add-sensible-rss-backoff

  Half-Shot

  committed
  92842fd

  Copy full SHA for 92842fd

• Remove testing URL

  Half-Shot

  committed
  5659d88

  Copy full SHA for 5659d88

• Add some variance to speed up while loop

  Half-Shot

  committed
  e6ddcb6

  Copy full SHA for e6ddcb6

• correct comment

  Half-Shot

  committed
  4838ba1

  Copy full SHA for 4838ba1

• Follow the advice and use a VecDeque as it's slightly faster.

  Half-Shot

  committed
  1614a2a

  Copy full SHA for 1614a2a

• Vastly better shuffle method

  Half-Shot

  committed
  958bdcb

  Copy full SHA for 958bdcb

• Speed up checking for previous guids.

  Half-Shot

  committed
  bec670f

  Copy full SHA for bec670f

• fix hasher function

  Half-Shot

  committed
  be4a468

  Copy full SHA for be4a468

• lint

  Half-Shot

  committed
  bd0822e

  Copy full SHA for bd0822e

• Content doesn't need to be calculated twice.

  Half-Shot

  committed
  db23748

  Copy full SHA for db23748

• Slightly more efficient iteration

  Half-Shot

  committed
  605139f

  Copy full SHA for 605139f

• Improve performance of backoff insertion

  Half-Shot

  committed
  ffd3881

  Copy full SHA for ffd3881

Commits on Feb 8, 2024

• Configure feed reader

  Half-Shot

  committed
  6649143

  Copy full SHA for 6649143

• lint

  Half-Shot

  committed
  149ca51

  Copy full SHA for 149ca51

• Ensure appending and removing from the queue works as expected.

  Half-Shot

  committed
  5a10c14

  Copy full SHA for 5a10c14

• Ensure we do keep urls that have been removed.

  Half-Shot

  committed
  251e80a

  Copy full SHA for 251e80a

Commits on Feb 9, 2024

• lint

  Half-Shot

  committed
  c67148c

  Copy full SHA for c67148c

• Inc/dec metrics as queue items are added/deleted.

  Half-Shot

  committed
  ce98771

  Copy full SHA for ce98771

• Add comment

  Half-Shot

  committed
  0bc49c2

  Copy full SHA for 0bc49c2

Commits on Feb 20, 2024

• tidy up

  Half-Shot

  committed
  60a3a78

  Copy full SHA for 60a3a78