De-dupe ReconnectApp in the persisted requests queue #47913

gedu · 2024-08-23T12:15:51Z

Details

we have 4 places that can trigger the ReconnectApp:

NetInfo.addEventListener
Middleware: recheckConnection
NetworkConnection.triggerReconnectionCallbacks(‘Pusher re-subscribed to private user channel’);
new one above: Interval recheckNetworkConnection

so, In the PersistedRequest file, within the save function, I would add a check to see if a ReconnectApp already exists in the persistedRequest queue. If it does, replace it with the new incoming ReconnectApp. This ensures we always maintain just one instance of it, with the most up-to-date information to send.

Fixed Issues

$ #47742
PROPOSAL: #47742 (comment)

Reference: #40059

Tests

Verify that no errors appear in the JS console

Base

Use the app, try to use it on different situations where you think calls to ReconnectApp can happen

With failing requests

Go to Settings -> Troubleshoot
Enable Simulate failing network requests
Go to Chat and send some messages or open different reports
Then go back to Settings -> Troubleshoot and disable Simulate failing network requests
Verify that one ReconnectApp shows up on the Network tab from DevTool

With failing requests

Go to Settings -> Troubleshoot
Enable Force offline
Go to Chat and send some messages or open different reports
Then go back to Settings -> Troubleshoot and disable Force offline
Verify that one ReconnectApp shows up on the Network tab from DevTool

Offline tests

QA Steps

Verify that no errors appear in the JS console

PR Author Checklist

Screenshots/Videos

Android: Native

single_reconnectApp_android.mp4

Android: mWeb Chrome

single_reconnectApp_androidWeb.mp4

iOS: Native

single_reconnectApp_ios.mp4

iOS: mWeb Safari

single_reconnectApp_iosWeb.mp4

MacOS: Chrome / Safari

single_reconnectApp_chrome.mp4

single_reconnectApp_safari.mp4

MacOS: Desktop

single_reconnectApp_desktop.mp4

melvin-bot · 2024-08-23T12:15:56Z

@sobitneupane Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

gedu · 2024-08-23T12:17:31Z

@roryabraham PR created, ready for review

roryabraham

I think we should:

create a more generalized framework for de-duping network requests in the request queue, as is done in this PR
add back all the other cases that were removed (i.e: if adding DeleteComment to the request queue, and there's any AddComment or UpdateComment for the same reportActionID, then just remove them all from the queue).
Add automated tests

Honestly, I was planning to do this myself because I have good historical context, but I just haven't prioritized it yet. I've finished up a lot of other stuff so might be able to proceed with #40059 now.

gedu · 2024-08-26T11:54:29Z

@roryabraham Nice, I like that idea of making it generic, I didn't know that other requests will need some strategy to fix it. Give that many requests require different strategies, for reconnectApp the structure you made for XXXXCommentdoesn't suit well. I would like to propose a single function that does what the request needs. Like checkAndFixConflictingRequest, which would return

{
  requests: [], // requests fixed
  persistAction: number, // if the requests sent can be saved into PersistedRequests
}

That way all is encapsulate into one place, into the request creation.
Maybe I can create the interface here, apply the reconnectApp strategy, add some tests. We can keep handling the XXXXComments in a separate PR

roryabraham · 2024-08-26T23:47:53Z

for reconnectApp the structure you made for XXXXCommentdoesn't suit well

Maybe I'm not seeing something, but I think it works fine for ReconnectApp:

For getConflictingRequests, return an array of any other serialized ReconnectApp calls. In practice there should only ever be 0 or 1 at a time.
For shouldSkipThisRequestOnConflict, return false because we do want the second ReconnectApp call to run.
For handleConflictingRequest, do nothing because ReconnectApp doesn't involve any optimistic data that needs to be cleaned up.

gedu · 2024-08-27T14:51:12Z

@roryabraham I don’t want to add extra functions that aren’t needed. As an example, bulkRemove won’t work for the ReconnectApp because it deletes the instance, and the new one gets added in a different position. I want to update the requests.

Let’s say we have 10 items in the queue, and there is a ReconnectApp at position 4; a new one will be added. The getConflictingRequests will return the item from position 4. If I delete it and add the new one, it will be added at the end, resulting in a different position.

I think the main logic from your PR can be inside the checkAndFixConflictingRequest (we can change the name), and it will work the same way.
This function will return a flag to check if it should save or update the request:

if (persistAction.type === CONST.SAVE) {
    PersistedRequests.save(request);
} else if (persistAction.type === CONST.UPDATE) {
    PersistedRequests.update(persistAction.index, request);
}

(I'm updating a bit while thinking and finding more cases)

gedu · 2024-08-28T14:53:23Z

@roryabraham in my last commit It is your code, from the PR, using just one single function, and you can see how reconnectApp will work. What do you think?

roryabraham

Overall this looks like a good implementation. It's an improvement over what I had in that it identifies the main use-cases we'll have and implements them with better guard rails. Good job 👍🏼

NAB but I kind of prefer replace and push as the actions instead of update and save, because they're a bit more distinctive from each other and better reflect the queue structure.

Also, I'd love to add some automated tests for this before merging it. Thanks!

src/libs/Network/SequentialQueue.ts

src/libs/actions/Report.ts

src/types/onyx/Request.ts

gedu · 2024-08-29T08:30:09Z

@roryabraham Great approach! It definitely makes sense. I will work on ReconnectApp with the new conflictResolver. This way, each change has its own PR, which makes it easier to track and identify any regressions. After it, we can proceed with looking into the DELETE_COMMENT. What do you think?
I will start working on tests and the suggested changes.

roryabraham · 2024-08-29T15:02:01Z

Sounds good 👍

gedu · 2024-08-30T15:09:23Z

@roryabraham I added a few tests, I'm giving a round of QA, what else I'm missing? what more tests do you think I should add?

# Conflicts: # src/libs/Network/SequentialQueue.ts

gedu · 2024-09-03T08:20:57Z

@roryabraham friendly ping

roryabraham

It's great that we've added tests to cover the underlying logic in SequentialQueue, but I think it's also important that we add test coverage for App.reconnectApp that ensures that our implementation of checkAndFixConflictingRequest is working as expected in that case.

src/types/onyx/Request.ts

tests/unit/SequentialQueueTest.ts

roryabraham · 2024-09-03T21:39:21Z

tests/unit/SequentialQueueTest.ts

+        expect(PersistedRequests.getLength()).toBe(2);
+    });
+
+    it('should push two requests with conflict resolution and replace', () => {


I think we should also add a test that queues other unrelated requests and make sure that the replaced request is in the same index as the conflicting ReconnectApp request.

i.e: [OpenReport, AddComment, ReconnectApp, OpenReport]

becomes [OpenReport, AddComment, ReconnectApp, OpenReport], with ReconnectApp still at index 2 for example

roryabraham · 2024-09-17T18:24:40Z

hmmm actually there's a problem with our current config. deprecation/deprecation can't be suppressed because it doesn't exist in the main eslint config. Working on a solution

roryabraham · 2024-09-17T18:46:50Z

Got it. @gedu you can fix the lint warnings on this branch by:

Merging main
Applying this diff (reasonable to suppress the lint errors in all these cases):

diff --git a/.eslintrc.js b/.eslintrc.js
index cb121953327..c9d11b97d7d 100644
--- a/.eslintrc.js
+++ b/.eslintrc.js
@@ -108,7 +108,7 @@ module.exports = {
         'plugin:you-dont-need-lodash-underscore/all',
         'plugin:prettier/recommended',
     ],
-    plugins: ['@typescript-eslint', 'jsdoc', 'you-dont-need-lodash-underscore', 'react-native-a11y', 'react', 'testing-library', 'eslint-plugin-react-compiler'],
+    plugins: ['@typescript-eslint', 'jsdoc', 'you-dont-need-lodash-underscore', 'react-native-a11y', 'react', 'testing-library', 'eslint-plugin-react-compiler', 'deprecation'],
     ignorePatterns: ['lib/**'],
     parser: '@typescript-eslint/parser',
     parserOptions: {
@@ -254,6 +254,7 @@ module.exports = {
                 },
             },
         ],
+        'deprecation/deprecation': 'off',
     },
 
     overrides: [
diff --git a/src/libs/ReportActionsUtils.ts b/src/libs/ReportActionsUtils.ts
index 3d0f3e1989a..842a51e69ed 100644
--- a/src/libs/ReportActionsUtils.ts
+++ b/src/libs/ReportActionsUtils.ts
@@ -213,8 +213,10 @@ function isActionOfType<T extends ReportActionName[]>(
 
 function getOriginalMessage<T extends ReportActionName>(reportAction: OnyxInputOrEntry<ReportAction<T>>): OriginalMessage<T> | undefined {
     if (!Array.isArray(reportAction?.message)) {
+        // eslint-disable-next-line deprecation/deprecation
         return reportAction?.message ?? reportAction?.originalMessage;
     }
+    // eslint-disable-next-line deprecation/deprecation
     return reportAction?.originalMessage;
 }
 
@@ -335,7 +337,7 @@ function isThreadParentMessage(reportAction: OnyxEntry<ReportAction>, reportID:
 /**
  * Returns the parentReportAction if the given report is a thread/task.
  *
- * @deprecated Use Onyx.connect() or withOnyx() instead
+ * @deprecated Use Onyx.connect() or useOnyx() instead
  */
 function getParentReportAction(report: OnyxInputOrEntry<Report>): OnyxEntry<ReportAction> {
     if (!report?.parentReportID || !report.parentReportActionID) {
@@ -1752,6 +1754,7 @@ export {
     getNumberOfMoneyRequests,
     getOneTransactionThreadReportID,
     getOriginalMessage,
+    // eslint-disable-next-line deprecation/deprecation
     getParentReportAction,
     getRemovedFromApprovalChainMessage,
     getReportAction,

gedu · 2024-09-18T13:28:17Z

After some new discussion around this, I would change a bit more the architecture

gedu · 2024-09-19T19:34:48Z

Creating some tests to reproduce a possible race condition and duplication. At least for now, I'm not able to reproduce the race condition:

[SequentialQueue] Unable to flush. Queue is paused.

    unpausing:  1726742359400

    DEBUG [SequentialQueue] Unpausing the queue and flushing 1003 requests

    pushing request with conflict resolution:  1726742359403

    [SequentialQueue] Conflict action for command ReconnectApp1: { type: 'replace', index: 1000 }

    going to update

    process

    process requestToProcess { command: 'OpenReport0' } 1726742359413

    persistedRequests {
        '0': { command: 'AddComment0' },
        '1': { command: 'OpenReport1' },
        [...],
        '998': { command: 'AddComment499' },
      '999': {
        command: 'ReconnectApp1',
        data: { accountID: 56789 },
        checkAndFixConflictingRequest: [Function: checkAndFixConflictingRequest]
      },
      '1000': { command: 'AddComment600' },
      '1001': { command: 'OpenReport600' }
    }
    ```
Ever conflict resolver and processing working good

gedu · 2024-09-24T07:36:39Z

Hey, I’m working on adding a new test for a new approach. Previously, ongoing requests weren’t saved into Onyx; they were just removed from the memory queue. Now, I’m adding a flag that allows you to decide per request whether to store it in Onyx or not. This is useful for requests we might want to process again if the user closes the app, while others, like OpenApp, don’t need to be saved because it will . I’m currently running some tests on this

…essing

# Conflicts: # .eslintrc.js # src/libs/Middleware/HandleUnusedOptimisticID.ts

# Conflicts: # src/libs/API/index.ts

# Conflicts: # src/libs/ReportActionsUtils.ts

gedu · 2024-09-26T13:25:30Z

@roryabraham can you take a look?

src/libs/Network/SequentialQueue.ts

src/libs/ReportActionsUtils.ts

src/libs/actions/PersistedRequests.ts

# Conflicts: # .eslintrc.js # src/libs/ReportActionsUtils.ts

github-actions · 2024-09-30T11:10:05Z

🧪🧪 Use the links below to test this adhoc build on Android, iOS, Desktop, and Web. Happy testing! 🧪🧪

Android 🤖	iOS 🍎
https://ad-hoc-expensify-cash.s3.amazonaws.com/android/47913/index.html	https://ad-hoc-expensify-cash.s3.amazonaws.com/ios/47913/index.html

Desktop 💻	Web 🕸️
https://ad-hoc-expensify-cash.s3.amazonaws.com/desktop/47913/NewExpensify.dmg	https://47913.pr-testing.expensify.com

👀 View the workflow run that generated this build 👀

roryabraham

Otherwise I think this prettymuch LGTM 👍🏼

.eslintrc.js

Updating reconnectApp from the persisted requests queue

e388597

gedu requested a review from a team as a code owner August 23, 2024 12:15

melvin-bot bot requested review from sobitneupane and removed request for a team August 23, 2024 12:15

roryabraham requested changes Aug 23, 2024

View reviewed changes

Merge branch 'main' into gedu/replace_reconnectapp_most_updated

6f278c0

roryabraham requested changes Aug 28, 2024

View reviewed changes

src/libs/Network/SequentialQueue.ts Outdated Show resolved Hide resolved

src/libs/actions/Report.ts Outdated Show resolved Hide resolved

src/libs/actions/Report.ts Outdated Show resolved Hide resolved

src/types/onyx/Request.ts Outdated Show resolved Hide resolved

Applying comments + unit tests

615a8ba

gedu force-pushed the gedu/replace_reconnectapp_most_updated branch from e915009 to 615a8ba Compare August 29, 2024 17:23

gedu added 4 commits August 29, 2024 19:33

Adding comment

8b505ab

Adding test to Sequential Queue

be72fec

Fixed TS error

6e40d66

removed the .only from test

31bf286

Merge branch 'main' into gedu/replace_reconnectapp_most_updated

7f1b772

# Conflicts: # src/libs/Network/SequentialQueue.ts

roryabraham requested changes Sep 3, 2024

View reviewed changes

gedu added 5 commits September 10, 2024 09:28

Fixes and moving the ongoing request to PersistedRequests

45bf5a6

Fixed APITest

3620da6

Merge branch 'main' into gedu/replace_reconnectapp_most_updated

1d9550c

Fixed tests and code

d07d912

Fixed lint issues

7170e14

Merge branch 'main' into gedu/replace_reconnectapp_most_updated

b9cb098

roryabraham changed the title ~~Updating reconnectApp from the persisted requests queue~~ De-dupe ReconnectApp in the persisted requests queue Sep 17, 2024

roryabraham mentioned this pull request Sep 17, 2024

De-dupe network requests #40059

Closed

49 tasks

roryabraham requested review from shubham1206agra and removed request for sobitneupane September 17, 2024 18:58

gedu added 3 commits September 17, 2024 21:45

disable deprecation fields

3b21e41

Removed hack, seems we don't need it anymore

919e254

Removed unused key

1c5c49b

gedu added 5 commits September 24, 2024 19:25

Added the chance to save the ongoing request into Onyx for later proc…

9ff5edd

…essing

Merge branch 'main' into gedu/replace_reconnectapp_most_updated

d3bf574

# Conflicts: # .eslintrc.js # src/libs/Middleware/HandleUnusedOptimisticID.ts

fixed eslint error

2bc9f32

Merge branch 'main' into gedu/replace_reconnectapp_most_updated

4b309bf

# Conflicts: # src/libs/API/index.ts

Merge branch 'main' into gedu/replace_reconnectapp_most_updated

cae7b49

# Conflicts: # src/libs/ReportActionsUtils.ts

mountiny added the Ready To Build label Sep 26, 2024

This comment has been minimized.

Sign in to view

roryabraham requested changes Sep 26, 2024

View reviewed changes

src/libs/Network/SequentialQueue.ts Outdated Show resolved Hide resolved

src/libs/Network/SequentialQueue.ts Outdated Show resolved Hide resolved

src/libs/ReportActionsUtils.ts Outdated Show resolved Hide resolved

src/libs/actions/PersistedRequests.ts Outdated Show resolved Hide resolved

gedu added 3 commits September 27, 2024 17:47

Merge branch 'main' into gedu/replace_reconnectapp_most_updated

2792cfe

# Conflicts: # .eslintrc.js # src/libs/ReportActionsUtils.ts

Fixed comments

703957d

replaced with unshift

7d90d66

roryabraham previously approved these changes Oct 1, 2024

View reviewed changes

.eslintrc.js Outdated Show resolved Hide resolved

roryabraham mentioned this pull request Oct 1, 2024

[$500] DeleteComment not working as expected #39432

Open

6 tasks

removed duplicated rule

1e0d4ed

gedu dismissed roryabraham’s stale review via 1e0d4ed October 1, 2024 09:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

De-dupe ReconnectApp in the persisted requests queue #47913

De-dupe ReconnectApp in the persisted requests queue #47913

gedu commented Aug 23, 2024

melvin-bot bot commented Aug 23, 2024

gedu commented Aug 23, 2024

roryabraham left a comment

gedu commented Aug 26, 2024 •

edited

Loading

roryabraham commented Aug 26, 2024 •

edited

Loading

gedu commented Aug 27, 2024 •

edited

Loading

gedu commented Aug 28, 2024

roryabraham left a comment

gedu commented Aug 29, 2024

roryabraham commented Aug 29, 2024

gedu commented Aug 30, 2024

gedu commented Sep 3, 2024

roryabraham left a comment

roryabraham Sep 3, 2024

roryabraham commented Sep 17, 2024

roryabraham commented Sep 17, 2024

gedu commented Sep 18, 2024

gedu commented Sep 19, 2024

gedu commented Sep 24, 2024

gedu commented Sep 26, 2024

This comment has been minimized.

github-actions bot commented Sep 30, 2024

roryabraham left a comment

De-dupe ReconnectApp in the persisted requests queue #47913

Are you sure you want to change the base?

De-dupe ReconnectApp in the persisted requests queue #47913

Conversation

gedu commented Aug 23, 2024

Details

Fixed Issues

Tests

Offline tests

QA Steps

PR Author Checklist

Screenshots/Videos

melvin-bot bot commented Aug 23, 2024

gedu commented Aug 23, 2024

roryabraham left a comment

Choose a reason for hiding this comment

gedu commented Aug 26, 2024 • edited Loading

roryabraham commented Aug 26, 2024 • edited Loading

gedu commented Aug 27, 2024 • edited Loading

gedu commented Aug 28, 2024

roryabraham left a comment

Choose a reason for hiding this comment

gedu commented Aug 29, 2024

roryabraham commented Aug 29, 2024

gedu commented Aug 30, 2024

gedu commented Sep 3, 2024

roryabraham left a comment

Choose a reason for hiding this comment

roryabraham Sep 3, 2024

Choose a reason for hiding this comment

roryabraham commented Sep 17, 2024

roryabraham commented Sep 17, 2024

gedu commented Sep 18, 2024

gedu commented Sep 19, 2024

gedu commented Sep 24, 2024

gedu commented Sep 26, 2024

This comment has been minimized.

github-actions bot commented Sep 30, 2024

roryabraham left a comment

Choose a reason for hiding this comment

gedu commented Aug 26, 2024 •

edited

Loading

roryabraham commented Aug 26, 2024 •

edited

Loading

gedu commented Aug 27, 2024 •

edited

Loading