refactor: switch from Axios for Fetch API #840

gauthier-th · 2024-06-25T00:33:07Z

Description

This PR switch our codebase from the Axios library to the native Fetch API instead.

Screenshot (if UI-related)

To-Dos

Successful build yarn build
Translation keys yarn i18n:extract
Database migration (if required)

Issues Fixed or Closed

Fixes #XXXX

Fallenbagel · 2024-06-28T18:04:49Z

src/components/Layout/UserWarnings/index.tsx

@@ -24,6 +24,7 @@ const UserWarnings: React.FC<UserWarningsProps> = ({ onClick }) => {
  let res = null;

  //check if a user has warnings
+  console.log(user);


Remove the console log

github-actions · 2024-06-29T22:40:07Z

This pull request has merge conflicts. Please resolve the conflicts so the PR can be successfully reviewed and merged.

Gauvino · 2024-07-04T23:41:33Z

Thank for the PR and after some test this is all the bugs I have encounter:

First: in movies and series when changing of sort type it break the image render

Second: when adding a filter half of the show or render

Third: can't comment on issue

Fourth: can't update a user in every section, only permissions one seem's to work, and for editing our account only notifications don't work

Fifth: when clicking on a movie, and clicking on a similar title it render with the image of the previous movie, also similar title movie don't render at all

Fallenbagel · 2024-07-05T05:16:20Z

@Gauvino your fourth bug, is not a bug. You can't save because of our current email validation. That user needs a proper email in their email field then you can save

Fallenbagel · 2024-07-09T14:06:19Z

We need help on this PR.

Running into an issue where every now and then on not that slow but somewhat slow Internet like 30mbps etc. This might be related to the bugs that @Gauvino posted on their comment.

Fetch request keeps timing out.

TypeError: fetch failed
    at node:internal/deps/undici/undici:12618:11
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async JellyfinAPI.get (/home/fallenbagel/code/jellyseerr/server/api/externalapi.ts:76:24)
    at async JellyfinAPI.getItemData (/home/fallenbagel/code/jellyseerr/server/api/jellyfin.ts:349:28)
    at async JellyfinScanner.processMovie (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:63:24)
    at async /home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:514:11
    at async Promise.all (index 18)
    at async JellyfinScanner.processItems (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:511:5)
    at async JellyfinScanner.loop (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:543:7)
    at async JellyfinScanner.run (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:657:11) {
  cause: ConnectTimeoutError: Connect Timeout Error
      at onConnectTimeout (node:internal/deps/undici/undici:7760:28)
      at node:internal/deps/undici/undici:7716:50
      at Immediate._onImmediate (node:internal/deps/undici/undici:7748:13)
      at processImmediate (node:internal/timers:478:21) {
    code: 'UND_ERR_CONNECT_TIMEOUT'
  }
}
Request took 10004ms

So we need input from someone who has any experience in solving this issue

M0NsTeRRR · 2024-07-09T22:10:08Z

We need help on this PR.

Running into an issue where every now and then on not that slow but somewhat slow Internet like 30mbps etc. This might be related to the bugs that @Gauvino posted on their comment.

Fetch request keeps timing out.

TypeError: fetch failed
    at node:internal/deps/undici/undici:12618:11
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async JellyfinAPI.get (/home/fallenbagel/code/jellyseerr/server/api/externalapi.ts:76:24)
    at async JellyfinAPI.getItemData (/home/fallenbagel/code/jellyseerr/server/api/jellyfin.ts:349:28)
    at async JellyfinScanner.processMovie (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:63:24)
    at async /home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:514:11
    at async Promise.all (index 18)
    at async JellyfinScanner.processItems (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:511:5)
    at async JellyfinScanner.loop (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:543:7)
    at async JellyfinScanner.run (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:657:11) {
  cause: ConnectTimeoutError: Connect Timeout Error
      at onConnectTimeout (node:internal/deps/undici/undici:7760:28)
      at node:internal/deps/undici/undici:7716:50
      at Immediate._onImmediate (node:internal/deps/undici/undici:7748:13)
      at processImmediate (node:internal/timers:478:21) {
    code: 'UND_ERR_CONNECT_TIMEOUT'
  }
}
Request took 10004ms

So we need input from someone who has any experience in solving this issue

Hello,

https://undici.nodejs.org/#/?id=network-address-family-autoselection
UND_ERR_CONNECT_TIMEOUT is an issue related to IPv6, not a slow connection according to the documentation. Additionally, 30mbps is not a slow connection. A timeout of 10004ms is expected to timeout. To test with a bad connection, open the network tab in your browser and set the speed limitation to 2G (40 kbit/s).

gauthier-th · 2024-07-09T22:36:51Z

We need help on this PR.

Running into an issue where every now and then on not that slow but somewhat slow Internet like 30mbps etc. This might be related to the bugs that @Gauvino posted on their comment.

Fetch request keeps timing out.
TypeError: fetch failed
    at node:internal/deps/undici/undici:12618:11
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async JellyfinAPI.get (/home/fallenbagel/code/jellyseerr/server/api/externalapi.ts:76:24)
    at async JellyfinAPI.getItemData (/home/fallenbagel/code/jellyseerr/server/api/jellyfin.ts:349:28)
    at async JellyfinScanner.processMovie (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:63:24)
    at async /home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:514:11
    at async Promise.all (index 18)
    at async JellyfinScanner.processItems (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:511:5)
    at async JellyfinScanner.loop (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:543:7)
    at async JellyfinScanner.run (/home/fallenbagel/code/jellyseerr/server/lib/scanners/jellyfin/index.ts:657:11) {
  cause: ConnectTimeoutError: Connect Timeout Error
      at onConnectTimeout (node:internal/deps/undici/undici:7760:28)
      at node:internal/deps/undici/undici:7716:50
      at Immediate._onImmediate (node:internal/deps/undici/undici:7748:13)
      at processImmediate (node:internal/timers:478:21) {
    code: 'UND_ERR_CONNECT_TIMEOUT'
  }
}
Request took 10004ms
So we need input from someone who has any experience in solving this issue
Hello,

https://undici.nodejs.org/#/?id=network-address-family-autoselection
UND_ERR_CONNECT_TIMEOUT is an issue related to IPv6, not a slow connection according to the documentation. Additionally, 30mbps is not a slow connection. A timeout of 10004ms is expected to timeout. To test with a bad connection, open the network tab in your browser and set the speed limitation to 2G (40 kbit/s).

Thanks for your response.
These issues are happening on the backend of the application, so it's not possible to debug with browser tools.

For IPv6, we tried things like adding --dns-result-order=ipv4first as argument.
I didn't manage to replicate this issue, but @Fallenbagel did so he may have other insights.

We'd be happy to try out other ideas.

M0NsTeRRR · 2024-07-09T22:52:58Z

Thanks for your feedback. dns-result-order should not help at all alone, as it only manages the DNS response order (IPv4 first doesn't prevent you from still getting both IPv4 and IPv6 results).

From the documentation if i'm not misleading the default configuration set getdefaultautoselectfamily to true which use first AAAA record first (if returned), first A, second AAAA, second A etc.

To confirm the IPv6 issue I see some possibilities to try (I'm not a Node.js expert, so I might miss some steps.) :

Force IPv6 connection with family in the TCP socket.
Using NODE_OPTIONS="--dns-result-order=ipv4first --no-network-family-autoselection" which should return IPv4 first and use the first returned record.
Disable IPv6 on the host to test
Fix IPv6 configuration on the host

Concerning the root cause, I suspect that Node.js detects your host supports IPv6, but it is misconfigured, which leads to this issue.

gauthier-th · 2024-07-10T13:34:20Z

@Gauvino we did some changes to try to fix this issue. Could you please try and see if it works now?

Gauvino · 2024-07-10T13:37:15Z

@Gauvino we did some changes to try to fix this issue. Could you please try and see if it works now?

Im gonna test this tonight, and will give feedback on everything !

Gauvino · 2024-07-10T16:41:56Z

Sorry for the news but nothing as change, all of the bugs are still there!

Does I can provide some logs are something to investigate more?

Fallenbagel

LGTM!

Fallenbagel · 2024-10-15T18:22:26Z

🎉 This PR is included in version 2.0.0 🎉

The release is available on:

v2.0.0
GitHub release

Your semantic-release bot 📦🚀

gauthier-th marked this pull request as ready for review June 28, 2024 15:12

gauthier-th requested a review from Fallenbagel as a code owner June 28, 2024 15:12

Fallenbagel reviewed Jun 28, 2024

View reviewed changes

github-actions bot added the merge conflict Cannot merge due to merge conflicts label Jun 29, 2024

gauthier-th force-pushed the switch-to-fetch-api branch from 67529df to b6c7c6f Compare June 30, 2024 15:27

github-actions bot removed the merge conflict Cannot merge due to merge conflicts label Jun 30, 2024

Fallenbagel added the help wanted Extra attention is needed label Jul 9, 2024

gauthier-th marked this pull request as draft July 9, 2024 14:17

gauthier-th added 12 commits July 11, 2024 15:48

refactor: switch ExternalAPI to Fetch API

99443d2

fix: add missing auth token in Plex request

bebbd69

fix: send proper URL params

a46cdb3

ci: try to fix format checker

c5e35aa

ci: ci: try to fix format checker

5865bfa

ci: try to fix format checker

543341d

refactor: make tautulli use the ExternalAPI class

1b2b137

refactor: add rate limit to fetch api

8f6cbef

refactor: add rate limit to fetch api

27cd125

refactor: switch server from axios to fetch api

0f6ba54

refactor: switch frontend from axios to fetch api

15568b1

fix: switch from URL objects to strings

6bcaa67

gauthier-th added 6 commits July 11, 2024 15:48

fix: use the right search params for ExternalAPI

faec326

fix: better log for ExternalAPI errors

943d84e

feat: add retry to external API requests

98c5500

fix: try to fix network errors with IPv6

c405453

fix: imageProxy rate limit

6d9a1c5

revert: remove retry to external API requests

eddbf2d

gauthier-th force-pushed the switch-to-fetch-api branch from 97c5e9c to eddbf2d Compare July 11, 2024 13:55

feat: set IPv4 first as an option

e9d6046

gauthier-th marked this pull request as ready for review July 11, 2024 14:38

gauthier-th added 2 commits July 13, 2024 02:03

fix(jellyfinapi): add missing argument in JellyfinAPI constructor

8be79fc

refactor: clean the rate limit utility

7528a7c

Fallenbagel approved these changes Jul 14, 2024

View reviewed changes

gauthier-th merged commit b36bb3f into develop Jul 14, 2024
9 checks passed

gauthier-th deleted the switch-to-fetch-api branch August 24, 2024 12:44

Fallenbagel added the released label Oct 15, 2024

M0NsTeRRR mentioned this pull request Oct 16, 2024

[Bug] CSRF Validation broken since 2.0.0 #1011

Closed

1 task

Fallenbagel mentioned this pull request Oct 16, 2024

fix(csrf): override fetch to handle requests with CSRF token #1014

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: switch from Axios for Fetch API #840

refactor: switch from Axios for Fetch API #840

gauthier-th commented Jun 25, 2024

Fallenbagel Jun 28, 2024

github-actions bot commented Jun 29, 2024

Gauvino commented Jul 4, 2024 •

edited

Loading

Fallenbagel commented Jul 5, 2024

Fallenbagel commented Jul 9, 2024

M0NsTeRRR commented Jul 9, 2024

gauthier-th commented Jul 9, 2024

M0NsTeRRR commented Jul 9, 2024 •

edited

Loading

gauthier-th commented Jul 10, 2024

Gauvino commented Jul 10, 2024

Gauvino commented Jul 10, 2024

Fallenbagel left a comment

Fallenbagel commented Oct 15, 2024

refactor: switch from Axios for Fetch API #840

refactor: switch from Axios for Fetch API #840

Conversation

gauthier-th commented Jun 25, 2024

Description

Screenshot (if UI-related)

To-Dos

Issues Fixed or Closed

Fallenbagel Jun 28, 2024

Choose a reason for hiding this comment

github-actions bot commented Jun 29, 2024

Gauvino commented Jul 4, 2024 • edited Loading

Fallenbagel commented Jul 5, 2024

Fallenbagel commented Jul 9, 2024

M0NsTeRRR commented Jul 9, 2024

gauthier-th commented Jul 9, 2024

M0NsTeRRR commented Jul 9, 2024 • edited Loading

gauthier-th commented Jul 10, 2024

Gauvino commented Jul 10, 2024

Gauvino commented Jul 10, 2024

Fallenbagel left a comment

Choose a reason for hiding this comment

Fallenbagel commented Oct 15, 2024

Gauvino commented Jul 4, 2024 •

edited

Loading

M0NsTeRRR commented Jul 9, 2024 •

edited

Loading