Duplicate G7 CGM Sensor Insert

[Knighth4wk]

Hello,

I am using xDrip+AAPS+NS setup. I recently updated to version 20240419 of xDrip and noticed mutiple duplicate entries every hour for CGM sensor start in my NS and AAPS treatments. AAPS is setup to receive BG data from xDrip and upload to NS.

I went through previous treatments entries and confirmed the duplicates started showing up right after updating xDrip to latest stable release.

Current xDrip Version: 20240419
Previous xDrip Version: 20230716
xDrip Data Source: G7 sensor
AAPS version: 3.2.0.4
Nightscout: 15.0.2
Phone: Samsung Galaxy S24 Ultra running Android 14

[Navid200]

xDrip has changed significantly for better performance with G7. You should use a more recent release as suggested in the guide: https://navid200.github.io/xDrip/docs/Dexcom/G7.html

Are you only uploading to Nightscout from AAPS and not also from xDrip?

[olorinmaia]

I've checked my son's AAPS and I notice the duplicate entries are there also. There are no duplicates in Nightscout however, therefor no duplicates in AAPS follow app, AAPSClient which is the app I mostly browse treatments history in, that's why it has been hard for me to discover this. Or if this has been happening earlier.

@Knighth4wk you are sure this didn't happen before 20140419 release? If so it is probably this commit 1d77e78 that has changed something within AAPS how the broadcast information is received. I've asked Milos for advice, I'll let you know what he says.

No duplicates in NS:

Duplicates in Treatments log > Careportal and User entry (They seem to be added every 5 min or recieved BG?):

User entry	Careportal

[Knighth4wk]

I have double checked NS, the duplicated entries are showing there too for me.

The previous version I was using 20230716 did not have this issue.

[olorinmaia]

Are you using NSClientV1 or NSClientV3?

[Knighth4wk]

I am using NSClientV3.

[olorinmaia]

NSClientV1 here. That could be the reason for why you get duplicates in NS and I don't. It's very hard for me to see in the logs why this is happening in AAPS, need help from @MilosKozak or someone more experienced. Here are logs from AAPS in a period this issue is occuring:
AndroidAPS_LOG_1717346137057.log.zip

[MilosKozak]

18:29:32.658 [DefaultDispatcher-worker-7] D/BGSOURCE: [XdripSourcePlugin$XdripSourceWorker.doWorkAndLog():102]: Received xDrip data: Bundle[{com.eveningoutpost.dexdrip.Extras.NoiseBlockLevel=200, com.eveningoutpost.dexdrip.Extras.BgSlope=7.348103020404346E-5, com.eveningoutpost.dexdrip.Extras.VersionInfo=2124052808 c67b74f-2024.05.28, com.eveningoutpost.dexdrip.Extras.Raw=0.0, com.eveningoutpost.dexdrip.Extras.Display.Units=mmol, com.eveningoutpost.dexdrip.Extras.Time=1717345763598, com.eveningoutpost.dexdrip.Extras.CalibrationInfo=-1, com.eveningoutpost.dexdrip.Extras.NsNoiseLevel=null, com.eveningoutpost.dexdrip.Extras.SensorStartedAt=1716739867561, com.eveningoutpost.dexdrip.Extras.Noise=-9999.0, com.eveningoutpost.dexdrip.Extras.SensorBattery=-1, com.eveningoutpost.dexdrip.Extras.BgSlopeName=FortyFiveUp, com.eveningoutpost.dexdrip.Extras.BgEstimate=150.0, com.eveningoutpost.dexdrip.Extras.SourceDesc=G6 Native / G5 Native, com.eveningoutpost.dexdrip.Extras.SourceInfo=G7, com.eveningoutpost.dexdrip.Extras.Collector.NanoStatus=  }]
18:34:33.391 [main] D/CORE: [DataReceiver.onReceive():35]: onReceive com.eveningoutpost.dexdrip.BgEstimate Bundle{ com.eveningoutpost.dexdrip.Extras.NoiseBlockLevel => 200; com.eveningoutpost.dexdrip.Extras.BgSlope => 0.0; com.eveningoutpost.dexdrip.Extras.VersionInfo => 2124052808 c67b74f-2024.05.28; com.eveningoutpost.dexdrip.Extras.Raw => 0.0; com.eveningoutpost.dexdrip.Extras.Display.Units => mmol; com.eveningoutpost.dexdrip.Extras.Time => 1717346064332; com.eveningoutpost.dexdrip.Extras.CalibrationInfo => -1; com.eveningoutpost.dexdrip.Extras.NsNoiseLevel => null; com.eveningoutpost.dexdrip.Extras.SensorStartedAt => 1716739868294; com.eveningoutpost.dexdrip.Extras.Noise => -9999.0; com.eveningoutpost.dexdrip.Extras.SensorBattery => -1; com.eveningoutpost.dexdrip.Extras.BgSlopeName => FortyFiveUp; com.eveningoutpost.dexdrip.Extras.BgEstimate => 150.0; com.eveningoutpost.dexdrip.Extras.SourceDesc => G6 Native / G5 Native; com.eveningoutpost.dexdrip.Extras.SourceInfo => G7; com.eveningoutpost.dexdrip.Extras.Collector.NanoStatus =>   ; }Bundle

Sensor start is received multiple times with different timestamp.
AAPS does deduplication on exact timestamp only

Here:
Sun May 26 2024 18:11:08 GMT+0200 (středoevropský letní čas)
Sun May 26 2024 18:11:07 GMT+0200 (středoevropský letní čas)

[Navid200]

@Knighth4wk I mentioned the concept of using the history in xDrip as a judgement mechanism. And I explained that going to a different xDrip, you may then get a different start time since the history remains with the previous xDrip and is not transferred to a new fresh install.

[Navid200]

Can AAPS be modified so that it does not trigger an alert if the sensor start time changes?

[Navid200]

If what matters to AAPS is the age of the sensor, AAPS can subtract the reported start time from the current time to come up with the sensor age.
This can be rounded to minutes and the result will never have any fluctuations.

Alternatively, we can add a new item to broadcast from xDrip that being the sensor age. But, I am not sure if I can get approval for it. I hope I can.
@jamorham

[Navid200]

The transmitter reports once every 5 minutes how long ago the session started. It does this using its internal clock. The error in the value that it reports is negligible (very small percentage).

The problem starts when we want to translate that into real (absolute) time.
Meaning instead of saying the session started 4.1 days ago, we say the session started on June 2 at 12:00:17.

Why do we need to do that?
Can you tell me what requirement exists to use absolute time in this case and why relative time will not work?

[Navid200]

Forgive me for post after post after post. But, I am not sure if we have a path forward. If every one of us assumes that the other person has to do something, this may never be resolved.

We have unintentionally created a vernier here: https://en.wikipedia.org/wiki/Vernier_scale

What the transmitter transmits is how long ago sensor started. We, xDrip, subtract that from the current time to calculate when exactly the sensor started. The accuracy of the result depends on the accuracy of both subtraction operands. The problem is that the relative time coming from the transmitter is based on the local oscillator of the transmitter while the current time is based on internet time. Those two time bases are drifting with respect to each other.
So, it is like someone is continuously sliding the vernier. The vernier reading is continuously being changed because of the drift.

Solution:
xDrip to broadcast the sensor age exactly as reported by the transmitter, rather than when it started. I (Navid) to submit a PR.
AAPS to use the incoming sensor age to do all its internal calculations.

@jamorham Is this acceptable to you?
@MilosKozak Would this be acceptable to you?