`OSConsistencyManager` & IAM fetch read-your-write consistency implementation #1486

rgomezp · 2024-09-27T21:31:52Z

Description

One Line Summary

Introduce and integrate a ConsistencyManager to manage read-your-write tokens for improved segment membership calculation.

Details

Motivation

This update introduces the ConsistencyManager to manage RYW tokens. The goal is to improve the accuracy of segment membership calculations by providing an open-ended & highly customizable blocking mechanism for operations that rely on having successfully synchronized client & server state.

For a first use-case, we want to block the fetching of IAMs until we have tokens for a user or subscription state update.

Scope

OSConsistencyManager implementation
Updates the Subscription and User backend services to return offsets needed by the OSConsistencyManager.
Modifies Subscription and UpdateUser operation executors to set tokens in the OSConsistencyManager.
Updates the OSMessagingController to delay IAM fetch until the token is available. Implements the retry logic.

Testing

Unit testing

New coverage for new ConsistencyManager class

Manual Testing

Set up iOS test app in OneSignal
Note the current app version (e.g. 1.0)
Create a segment targeting a specific version (e.g. 1.1)
Create an in-app using the new segment
Open the app (see you don't get an in-app)
Close the app
Change the version to the one in the segment (e.g. 1.1)
Open the app (see you get the in-app)

Affected code checklist

Checklist

Overview

I have filled out all REQUIRED sections above
PR does one thing
- If it is hard to explain how any codes changes are related to each other then it most likely needs to be more than one PR
Any Public API changes are explained in the PR details and conform to existing APIs

Testing

I have included test coverage for these changes, or explained why they are not needed
All automated tests pass, or I explained why that is not possible
I have personally tested this on my device, or explained why that is not possible

Final pass

Code is as readable as possible.
- Simplify with less code, followed by splitting up code into well named functions and variables, followed by adding comments to the code.
I have reviewed this PR myself, ensuring it meets each checklist item
- WIP (Work In Progress) is ok, but explain what is still in progress and what you would like feedback on. Start the PR title with "WIP" to indicate this.

This change is

nan-li · 2024-10-10T16:16:38Z

iOS_SDK/OneSignalSDK/OneSignalOSCore/Source/Consistency/OSConsistencyManager.swift

+    private override init() {}
+
+    // Function to set the token in a thread-safe manner
+    @objc public func setRywToken(id: String, key: OSIamFetchOffsetKey, value: String?) {


The only expected usage of this method is for IAM Fetch?

The entire class can be used in the future for other RYW consistency projects!

Ok, I wanted to clarify as the parameter is explicitly OSIamFetchOffsetKey

Whoops that's a mistake

iOS_SDK/OneSignalSDK/OneSignalOSCore/Source/Consistency/OSConsistencyManager.swift

iOS_SDK/OneSignalSDK/OneSignalOSCore/Source/Consistency/IamFetch/OSIamFetchReadyCondition.swift

iOS_SDK/OneSignalSDK/OneSignalInAppMessages/Controller/OSMessagingController.m

iOS_SDK/OneSignalSDK/OneSignalUser/Source/Executors/OSSubscriptionOperationExecutor.swift

iOS_SDK/OneSignalSDK/OneSignalUser/Source/OSSubscriptionModelStoreListener.swift

iOS_SDK/OneSignalSDK/OneSignalCore/Source/API/OneSignalClient.m

iOS_SDK/OneSignalSDK/OneSignalOutcomes/Source/OutcomeEvents/OSSessionManager.h

iOS_SDK/OneSignalSDK/Source/OneSignalTracker.m

iOS_SDK/OneSignalSDK/OneSignalOSCore/Source/Consistency/OSCondition.swift

Motivation: encapsulating object to keep rywTokens & rywDelays together Override equality

Motivation: Manages (kafka) offsets that function as read-your-write tokens for more accurate segment membership calculation. The manager works based on conditions & offsets. Offsets are stored in a nested map indexed by a unique id (e.g. `onesignalId`) and offset key (e.g. `USER_UPDATE`). This allows us to track offsets on a per-user basis (e.g. handle switching users). Conditions work by creating a blocking mechanism with customizable offset retrieval until a pre-defined condition is met (e.g. at least two specific offsets are available). OSCondition interface: allows extensibility for future applications to control offset blocking mechanism in consistency use-cases.

Motivation: custom condition to block offset retrieval until condition is met: user & subscription offsets are both available or just the user offset is available. This is because we always make a user update call (update the session count) but not always for subscription.

Motivation: commit new file references

…ager` Motivation: move into `OSSessionManager` to be accessible from other modules

Motivation: we need to pass the offset, session duration, & retry count to the GET IAM request.

Motivation: retry logic should work as follows: 1. initial request with offset --> failure 2. get retry limit & retry after from response & start retrying 3. if hit limit, make one final request with the offset: 0 The final request tells the backend, just show me what you got. Example: If the retry limit is 3 & we never get a successful response retrying, we should end up with 5 total requests.

Motivation: executors make the update or create requests and get back an offset which is saved in the consistency manager

Motivation: we need to flush the delta queue on start in order to immediately trigger updates that would return a RYW token so we can unblock IAM fetch as soon as possible.

Created a new target OneSignalOSCoreTests

Motivation: if we have a subscription update enqueued, we need to consider it. Otherwise we should just consider the userUpdateTokenSet.

Motivation: fix all lint issues

Motivation: On fresh installs, we don't yet have a user or a subscription to update so we wouldn't get a RYW token until those requests are executed (5 seconds later when the operations are processed)

rgomezp changed the title ~~Consistency manager~~ OSConsistencyManager & IAM fetch read-your-write consistency implementation Sep 27, 2024

rgomezp force-pushed the consistency-manager branch 3 times, most recently from 4fbe142 to 7f07d0a Compare September 30, 2024 17:53

rgomezp requested review from nan-li and emawby September 30, 2024 17:55

rgomezp force-pushed the consistency-manager branch 4 times, most recently from ebcda07 to 8be39ed Compare October 3, 2024 19:50

rgomezp force-pushed the consistency-manager branch 7 times, most recently from 6dbf0ca to d546445 Compare October 10, 2024 22:17

nan-li reviewed Oct 11, 2024

View reviewed changes

rgomezp requested a review from nan-li October 15, 2024 18:32

rgomezp force-pushed the consistency-manager branch from d26ad02 to 10e877b Compare October 15, 2024 20:56

rgomezp mentioned this pull request Oct 15, 2024

Iam/remove iam caching #1499

Merged

18 tasks

rgomezp force-pushed the consistency-manager branch 2 times, most recently from d5540ae to 9c404e7 Compare October 16, 2024 19:10

rgomezp removed the request for review from emawby October 16, 2024 20:32

rgomezp force-pushed the consistency-manager branch from 9c404e7 to 66dc1cd Compare October 16, 2024 23:12

OSReadYourWriteData Implementation

fa3c750

Motivation: encapsulating object to keep rywTokens & rywDelays together Override equality

rgomezp force-pushed the consistency-manager branch from 66dc1cd to 279cce6 Compare October 18, 2024 22:48

nan-li approved these changes Oct 23, 2024

View reviewed changes

rgomezp added 2 commits October 23, 2024 14:17

rgomezp added 10 commits October 23, 2024 14:17

Update project.pbxproj

6f40409

Motivation: commit new file references

Move getTimeFocusedElapsed from OneSignalTracker to `OSSessionMan…

b07df39

…ager` Motivation: move into `OSSessionManager` to be accessible from other modules

Modify OSRequestGetInAppMessages method to include new header values

e7d65f6

Motivation: we need to pass the offset, session duration, & retry count to the GET IAM request.

Update executors to set offsets on consistency manager

d09d464

Motivation: executors make the update or create requests and get back an offset which is saved in the consistency manager

Flush delta queue on start

924d8a1

Motivation: we need to flush the delta queue on start in order to immediately trigger updates that would return a RYW token so we can unblock IAM fetch as soon as possible.

Update UnitTestApp TestPlan xctestplan

b8aeed2

Created a new target OneSignalOSCoreTests

Update OSIamFetchReadyCondition to sharedInstance to waist on sub update

82e468b

Motivation: if we have a subscription update enqueued, we need to consider it. Otherwise we should just consider the userUpdateTokenSet.

Update OneSignalClient to include headers

4f7a9dd

Run swiftlint --fix

663edc1

Motivation: fix all lint issues

rgomezp force-pushed the consistency-manager branch from 32cf760 to 663edc1 Compare October 23, 2024 19:22

Clarifying comment re: why we track RYW tokens for user create as well

75eab0f

Motivation: On fresh installs, we don't yet have a user or a subscription to update so we wouldn't get a RYW token until those requests are executed (5 seconds later when the operations are processed)

nan-li approved these changes Oct 23, 2024

View reviewed changes

rgomezp merged commit 66db67e into main Oct 23, 2024
2 of 4 checks passed

rgomezp deleted the consistency-manager branch October 23, 2024 21:29

This was referenced Oct 24, 2024

Release 5.2.6 #1503

Merged

Release 5.2.6 OneSignal/OneSignal-XCFramework#103

Merged

This was referenced Nov 5, 2024

Release 5.2.7 OneSignal/OneSignal-Cordova-SDK#1029

Merged

Release 5.2.6 OneSignal/react-native-onesignal#1757

Merged

Release 5.2.7 OneSignal/OneSignal-Flutter-SDK#969

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`OSConsistencyManager` & IAM fetch read-your-write consistency implementation #1486

`OSConsistencyManager` & IAM fetch read-your-write consistency implementation #1486

rgomezp commented Sep 27, 2024 •

edited

Loading

nan-li Oct 10, 2024

rgomezp Oct 14, 2024 •

edited

Loading

nan-li Oct 15, 2024

rgomezp Oct 16, 2024

OSConsistencyManager & IAM fetch read-your-write consistency implementation #1486

OSConsistencyManager & IAM fetch read-your-write consistency implementation #1486

Conversation

rgomezp commented Sep 27, 2024 • edited Loading

Description

One Line Summary

Details

Motivation

Scope

Testing

Unit testing

Manual Testing

Affected code checklist

Checklist

Overview

Testing

Final pass

nan-li Oct 10, 2024

Choose a reason for hiding this comment

rgomezp Oct 14, 2024 • edited Loading

Choose a reason for hiding this comment

nan-li Oct 15, 2024

Choose a reason for hiding this comment

rgomezp Oct 16, 2024

Choose a reason for hiding this comment

`OSConsistencyManager` & IAM fetch read-your-write consistency implementation #1486

`OSConsistencyManager` & IAM fetch read-your-write consistency implementation #1486

rgomezp commented Sep 27, 2024 •

edited

Loading

rgomezp Oct 14, 2024 •

edited

Loading