Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replay time ordering #405

Merged
merged 28 commits into from
Dec 9, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
83ace70
Include locationMs, and replayRequestMs
lmeier Oct 21, 2024
233ccc2
Ensure updatedAtMsDiff is correct
lmeier Oct 21, 2024
3b84c99
bump version
lmeier Oct 21, 2024
2e7bbbd
fix ordering
lmeier Oct 21, 2024
109283d
correct version
lmeier Oct 21, 2024
9ee710f
Add logging of visits
lmeier Oct 23, 2024
dcd18b8
simplify to just updatedAtMsDiff
lmeier Oct 25, 2024
531feb1
fix lint error
lmeier Oct 25, 2024
ce6fb3a
always include updatedAtMsDiff
lmeier Oct 25, 2024
19dfb83
log cleanup
lmeier Oct 28, 2024
b6a5ee8
Merge branch 'master' into replayTimestamps
lmeier Oct 28, 2024
cc8b817
timeInMs --> locationMs, don't send updatedAtMsDiff if verified
lmeier Nov 7, 2024
161cba4
extra bracket
lmeier Nov 7, 2024
8d7bdd6
nit
lmeier Nov 7, 2024
a214c59
Merge branch 'master' into replayTimestamps
KennyHuRadar Nov 7, 2024
2d22f43
not verified or not foreground
lmeier Nov 7, 2024
9b1432c
version bump
KennyHuRadar Nov 7, 2024
a3818cf
don't conditionalize for verified
lmeier Nov 8, 2024
63d548d
Revert "log cleanup"
lmeier Nov 8, 2024
04bae9e
don't conditionalize updatedAtMsDiff at all
lmeier Nov 8, 2024
910d489
make api helper logic more clear
lmeier Nov 8, 2024
ad35612
Merge branch 'master' into replayTimestamps
KennyHuRadar Nov 11, 2024
1cd42d9
Reformat to more closely match android impl
lmeier Dec 5, 2024
b138b15
Make nil check explict
lmeier Dec 5, 2024
1f5c6d0
put behind a feature setting
lmeier Dec 6, 2024
1c7ca58
Typo
lmeier Dec 6, 2024
d3c24f7
Merge branch 'master' into replayTimestamps
KennyHuRadar Dec 9, 2024
f06ebe5
bump version
KennyHuRadar Dec 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions RadarSDK/RadarAPIClient.m
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,7 @@ - (void)trackWithLocation:(CLLocation *_Nonnull)location
return completionHandler(RadarStatusErrorPublishableKey, nil, nil, nil, nil, nil, nil);
}
NSMutableDictionary *params = [NSMutableDictionary new];
RadarSdkConfiguration *sdkConfiguration = [RadarSettings sdkConfiguration];
BOOL anonymous = [RadarSettings anonymousTrackingEnabled];
params[@"anonymous"] = @(anonymous);
if (anonymous) {
Expand Down Expand Up @@ -251,10 +252,11 @@ - (void)trackWithLocation:(CLLocation *_Nonnull)location
params[@"floorLevel"] = @(location.floor.level);
}
long nowMs = (long)([NSDate date].timeIntervalSince1970 * 1000);
if (!foreground) {
long timeInMs = (long)(location.timestamp.timeIntervalSince1970 * 1000);
params[@"updatedAtMsDiff"] = @(nowMs - timeInMs);
long locationMs = (long)(location.timestamp.timeIntervalSince1970 * 1000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok so I guess the approach now is:
• change foreground = true to also use updatedAtMsDiff (for consistency? I might have been the one to bring this up?)
• Don't special case verified = true on the client side, handle it differently on the server side

If we decided not to change foreground = true to updatedAtMsDiff, would the other changes to updatedAtMsDiff in these PRs be enough to fix the bug? Or is the inconsistent behavior between foreground = true and foreground = false causing a problem?

Copy link
Contributor Author

@lmeier lmeier Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 change foreground = true to also use updatedAtMsDiff (for consistency? I might have been the one to bring this up?)

If we decided not to change foreground = true to updatedAtMsDiff, would the other changes to updatedAtMsDiff in these PRs be enough to fix the bug? Or is the inconsistent behavior between foreground = true and foreground = false causing a problem?

Yes, for three reasons:

  • not using updatedAtMsDiff causes a problem when a foreground location gets forced to server time while background locations are adjusted per updatedAtMsDiff, leading to inconsistent ordering (a subsequent background location can be brought erroneously before the foreground location by the updatedAtMsDiff adjustment). So no, the other updatedAtMsDiff changes would be not sufficient to fully fix the bug.
  • for consistency
  • because there's no empirical reason to specially trust foreground locations from the OS as time-correct. It's true that foreground locations aren't subject to delays related to backgrounding, but the OS can and does deliver locations when foregrounded that have a non now timestamp

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

• Don't special case verified = true on the client side, handle it differently on the server side

Correct, per slack conversation. I'll link in #mobile

if (sdkConfiguration.useForegroundLocationUpdatedAtMsDiff || !foreground) {
params[@"updatedAtMsDiff"] = @(nowMs - locationMs);
}
params[@"locationMs"] = @(locationMs);
params[@"foreground"] = @(foreground);
params[@"stopped"] = @(stopped);
params[@"replayed"] = @(replayed);
Expand Down Expand Up @@ -298,7 +300,6 @@ - (void)trackWithLocation:(CLLocation *_Nonnull)location
[tripParams setValue:[Radar stringForMode:tripOptions.mode] forKey:@"mode"];
params[@"tripOptions"] = tripParams;
}

RadarTrackingOptions *options = [Radar getTrackingOptions];
if (options.syncGeofences) {
params[@"nearbyGeofences"] = @(YES);
Expand Down Expand Up @@ -340,7 +341,6 @@ - (void)trackWithLocation:(CLLocation *_Nonnull)location
}
}
params[@"appId"] = [[NSBundle mainBundle] bundleIdentifier];
RadarSdkConfiguration *sdkConfiguration = [RadarSettings sdkConfiguration];
if (sdkConfiguration.useLocationMetadata) {
NSMutableDictionary *locationMetadata = [NSMutableDictionary new];
locationMetadata[@"motionActivityData"] = [RadarState lastMotionActivityData];
Expand Down Expand Up @@ -415,9 +415,6 @@ - (void)trackWithLocation:(CLLocation *_Nonnull)location
// create a copy of params that we can use to write to the buffer in case of request failure
NSMutableDictionary *bufferParams = [params mutableCopy];
bufferParams[@"replayed"] = @(YES);
bufferParams[@"updatedAtMs"] = @(nowMs);
// remove the updatedAtMsDiff key because for replays we want to rely on the updatedAtMs key for the time instead
[bufferParams removeObjectForKey:@"updatedAtMsDiff"];

[[RadarReplayBuffer sharedInstance] writeNewReplayToBuffer:bufferParams];
} else if (options.replay == RadarTrackingOptionsReplayStops && stopped &&
Expand Down
32 changes: 31 additions & 1 deletion RadarSDK/RadarAPIHelper.m
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,39 @@ - (void)requestWithMethod:(NSString *)method
}

if (params) {
[req setHTTPBody:[NSJSONSerialization dataWithJSONObject:params options:0 error:NULL]];
NSNumber *prevUpdatedAtMsDiff = params[@"updatedAtMsDiff"];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that we're collecting locationMS we could also pass up updatedAt (we probably already are) we could do this bookkeeping server side but I think let's do that in a v2 in the interest of getting this out.

NSArray *replays = params[@"replays"];
if (prevUpdatedAtMsDiff || replays) {
NSMutableDictionary *requestParams = [params mutableCopy];
long nowMs = (long)([NSDate date].timeIntervalSince1970 * 1000);
NSNumber *locationMs = params[@"locationMs"];

if (locationMs != nil && prevUpdatedAtMsDiff != nil) {
long updatedAtMsDiff = nowMs - [locationMs longValue];
requestParams[@"updatedAtMsDiff"] = @(updatedAtMsDiff);
}

if (replays) {
NSMutableArray *updatedReplays = [NSMutableArray arrayWithCapacity:replays.count];
for (NSDictionary *replay in replays) {
NSMutableDictionary *updatedReplay = [replay mutableCopy];
NSNumber *replayLocationMs = replay[@"locationMs"];
if (replayLocationMs != nil) {
long replayUpdatedAtMsDiff = nowMs - [replayLocationMs longValue];
updatedReplay[@"updatedAtMsDiff"] = @(replayUpdatedAtMsDiff);
}
[updatedReplays addObject:updatedReplay];
}
requestParams[@"replays"] = updatedReplays;
}

[req setHTTPBody:[NSJSONSerialization dataWithJSONObject:requestParams options:0 error:NULL]];
} else {
[req setHTTPBody:[NSJSONSerialization dataWithJSONObject:params options:0 error:NULL]];
}
}


NSURLSessionConfiguration *configuration;
if (extendedTimeout) {
configuration = [NSURLSessionConfiguration defaultSessionConfiguration];
Expand Down
4 changes: 4 additions & 0 deletions RadarSDK/RadarLocationManager.m
Original file line number Diff line number Diff line change
Expand Up @@ -1131,6 +1131,10 @@ - (void)locationManager:(CLLocationManager *)manager didVisit:(CLVisit *)visit {
return;
}

[[RadarLogger sharedInstance] logWithLevel:RadarLogLevelDebug
message:[NSString stringWithFormat:@"Visit detected | arrival = %@; departure = %@; horizontalAccuracy = %f; visit.coordinate = (%f, %f); manager.location = %@",
visit.arrivalDate, visit.departureDate, visit.horizontalAccuracy, visit.coordinate.latitude, visit.coordinate.longitude, manager.location]];

BOOL tracking = [RadarSettings tracking];
if (!tracking) {
[[RadarLogger sharedInstance] logWithLevel:RadarLogLevelDebug message:@"Ignoring visit: not tracking"];
Expand Down
3 changes: 3 additions & 0 deletions RadarSDK/RadarSdkConfiguration.h
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ NS_ASSUME_NONNULL_BEGIN
@property (nonatomic, assign) BOOL useLocationMetadata;

@property (nonatomic, assign) BOOL useOpenedAppConversion;

@property (nonatomic, assign) BOOL useForegroundLocationUpdatedAtMsDiff;

/**
Initializes a new RadarSdkConfiguration object with given value.
*/
Expand Down
13 changes: 10 additions & 3 deletions RadarSDK/RadarSdkConfiguration.m
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,16 @@ - (instancetype)initWithDict:(NSDictionary *)dict {
_useLocationMetadata = [(NSNumber *)useLocationMetadataObj boolValue];
}

NSObject *useOpenedAppConversion = dict[@"useOpenedAppConversion"];
NSObject *useOpenedAppConversionObj = dict[@"useOpenedAppConversion"];
_useOpenedAppConversion = NO;
if (useOpenedAppConversion && [useOpenedAppConversion isKindOfClass:[NSNumber class]]) {
_useOpenedAppConversion = [(NSNumber *)useOpenedAppConversion boolValue];
if (useOpenedAppConversionObj && [useOpenedAppConversionObj isKindOfClass:[NSNumber class]]) {
_useOpenedAppConversion = [(NSNumber *)useOpenedAppConversionObj boolValue];
}

NSObject *useForegroundLocationUpdatedAtMsDiffObj = dict[@"foregroundLocationUseUpdatedAtMsDiff"];
_useForegroundLocationUpdatedAtMsDiff = NO;
if (useForegroundLocationUpdatedAtMsDiffObj && [useForegroundLocationUpdatedAtMsDiffObj isKindOfClass:[NSNumber class]]) {
_useForegroundLocationUpdatedAtMsDiff = [(NSNumber *)useForegroundLocationUpdatedAtMsDiffObj boolValue];
}

return self;
Expand All @@ -90,6 +96,7 @@ - (NSDictionary *)dictionaryValue {
dict[@"useRadarModifiedBeacon"] = @(_useRadarModifiedBeacon);
dict[@"useLocationMetadata"] = @(_useLocationMetadata);
dict[@"useOpenedAppConversion"] = @(_useOpenedAppConversion);
dict[@"useForegroundLocationUpdatedAtMsDiff"] = @(_useForegroundLocationUpdatedAtMsDiff);

return dict;
}
Expand Down