New function `merge_camtrapdp()` #112

sannegovaert · 2024-07-25T13:23:49Z

Remarks

Assume that locationID's and individualID's must not be unique as the same location or individual can be used in different data packages.
Assume that duplicatesd ID's are between packages, not within.
If duplicates are present, all values of that identifier get a prefix.

Helper functions
I created 6 helper functions. Maybe they can be simplified or reduced in number!
They now live at utils.R.

function	functionality in `merge_camtrapdp()`	suggestion
`check_duplicate_ids()`	add prefix to identifiers with duplicates	helper function @ utils.R
`add_prefix()`	add prefix to identifiers with duplicates	helper function @ utils.R
`normalize_list()`	remove duplicated elements of lists	helper function @ utils.R or separate file.R
`is_subset()`	remove duplicated elements of lists	helper function @ utils.R or separate file.R
`update_unique()`	remove duplicated elements of lists	helper function @ utils.R or separate file.R
`remove_duplicates()`	remove duplicated elements of lists	helper function @ utils.R or separate file.R

codecov · 2024-07-25T13:27:40Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.91%. Comparing base (4e52a04) to head (5e7360f).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #112      +/-   ##
==========================================
+ Coverage   99.89%   99.91%   +0.01%     
==========================================
  Files          23       24       +1     
  Lines         983     1211     +228     
==========================================
+ Hits          982     1210     +228     
  Misses          1        1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…'s and observationID's"

…nerate_deploymentID()`

PietrH · 2024-11-11T13:48:06Z

R/utils.R

+    eventID = FALSE)
+
+  deploymentIDs <- c(
+    unique(purrr::pluck(deployments(x), "deploymentID")),


Can deploymentID be missing from deployments(x) ? If not, then purrr::chuck() will throw an error if it is missing, thus making the line a little bit more defensive.

Suggested change

unique(purrr::pluck(deployments(x), "deploymentID")),

unique(purrr::chuck(deployments(x), "deploymentID")),

PietrH · 2024-11-11T13:48:21Z

R/utils.R

+
+  deploymentIDs <- c(
+    unique(purrr::pluck(deployments(x), "deploymentID")),
+    unique(purrr::pluck(deployments(y), "deploymentID"))


Suggested change

unique(purrr::pluck(deployments(y), "deploymentID"))

unique(purrr::chuck(deployments(y), "deploymentID"))

PietrH · 2024-11-11T13:48:41Z

R/utils.R

+    unique(purrr::pluck(media(x), "mediaID")),
+    unique(purrr::pluck(media(y), "mediaID"))


Suggested change

unique(purrr::pluck(media(x), "mediaID")),

unique(purrr::pluck(media(y), "mediaID"))

unique(purrr::chuck(media(x), "mediaID")),

unique(purrr::chuck(media(y), "mediaID"))

PietrH · 2024-11-11T13:48:54Z

R/utils.R

+    unique(purrr::pluck(observations(x), "observationID")),
+    unique(purrr::pluck(observations(y), "observationID"))


Suggested change

unique(purrr::pluck(observations(x), "observationID")),

unique(purrr::pluck(observations(y), "observationID"))

unique(purrr::chuck(observations(x), "observationID")),

unique(purrr::chuck(observations(y), "observationID"))

PietrH · 2024-11-11T13:49:08Z

R/utils.R

+    unique(purrr::pluck(media(x), "eventID")),
+    unique(purrr::pluck(media(y), "eventID"))


Suggested change

unique(purrr::pluck(media(x), "eventID")),

unique(purrr::pluck(media(y), "eventID"))

unique(purrr::chuck(media(x), "eventID")),

unique(purrr::chuck(media(y), "eventID"))

PietrH · 2024-11-11T13:50:11Z

R/utils.R

+  )
+
+  # Check for duplicates
+  if (any(duplicated(deploymentIDs))) {result$deploymentID <- TRUE}


anyDuplicated() ?

PietrH · 2024-11-11T13:51:57Z

R/utils.R

+    # Add unique resources from y
+    y_unique_resources <-
+      y_additional_resources[!y_additional_resources %in% duplicated_names]
+    purrr::map(y_unique_resources, function(resource_name) {


Could purrr::imap() make this a bit easier instead of having to create the index in the anonymous function?

PietrH · 2024-11-11T13:55:09Z

R/utils.R

+  unique_data_list <-
+    purrr::map(unique_data, function(x) {
+      x <- as.list(x)
+      x[!sapply(x, is.na)]


sapply() is not type safe, and will set off ropensci review alarms!

PietrH · 2024-11-11T13:55:55Z

I only had a brief look, but an more in depth review of Peter is coming up.

sannegovaert added 10 commits July 23, 2024 17:07

new function merge_camtrapdp()

ffa9ee6

Create test-merge_camtrapdp.R

0b677c5

document()

10d9227

add test "merge_camtrapdp() returns no duplicated deploymentID's"

04afa53

fix typo

2df5162

import digest

d8f6f0a

update test

2ebdcc0

give unique deploymentIDs to deployments

bcea3dc

set unique mediaID's and observationID's

82fa057

update comment

429f201

sannegovaert added 19 commits July 25, 2024 15:35

update documentation

8ce8ad2

update examples

9297cd9

correct typo

fefac69

delete space in messages

e80eb49

test merge_camtrapdp() returns message when ID's are replaced

298b459

update test "merge_camtrapdp() returns unique deplpymentID's, mediaID…

61bb20d

…'s and observationID's"

character limit

53eaded

create helper function replace_duplicated_deploymentID()

03051d9

create helper function vdigest_crc32()

af034cf

rename and update function replace_duplicated_deploymentID() to `ge…

279faaa

…nerate_deploymentID()`

update comments

2c6c71f

new helper function generate_mediaID()

585d14e

add helper function generate_observationID()

5f007fb

update generate_observationID() to replace_observationID()

94f3c36

update generate_mediaID() to replace_mediaID()

0924691

update generate_deploymentID() to replace_deploymentID()

40e0a14

new helper function replace_duplicatedIDs()

f6caca2

update comment

b533e4e

check for length ID

63bb091

sannegovaert added 10 commits October 16, 2024 18:12

add documentation

f8327b6

update on project(s)

99ebe0d

use x and y instead of x1 and x2

637ee27

fix example

0ed0511

add visible binding for global variables

e166bb6

typo

eea76b6

document()

ccc479b

Update DESCRIPTION

800a49c

avoid error on lacking visible binding

e86a7b7

undo mistake

dce3167

sannegovaert requested a review from peterdesmet October 17, 2024 08:46

peterdesmet marked this pull request as ready for review October 17, 2024 12:28

peterdesmet and others added 6 commits October 17, 2024 14:31

Merge branch 'main' into merge_datasets

74983fd

Merge branch 'main' into merge_datasets

e659911

check for additional resources

b959be5

add additional resources

f3c6633

move to helper functions

82d2d26

typo's

19f98bd

sannegovaert force-pushed the merge_datasets branch from 9da984f to 19f98bd Compare October 25, 2024 18:24

sannegovaert and others added 9 commits October 25, 2024 22:06

Update test-merge_camtrapdp.R

4451fee

fix merge_additional_resources()

427226f

update documentation

f3f2a45

reorder

28802d3

Add new helper function

fd068f9

Replace NULL values (generated because of reading JSON) with NA

572d5af

Update test-write_camtrapdp.R

84dc619

Use resources()

7adf2c3

avoid tidyselect warning

5e7360f

PietrH requested changes Nov 11, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New function `merge_camtrapdp()` #112

New function `merge_camtrapdp()` #112

sannegovaert commented Jul 25, 2024 •

edited

Loading

codecov bot commented Jul 25, 2024 •

edited

Loading

PietrH Nov 11, 2024

PietrH Nov 11, 2024

PietrH Nov 11, 2024

PietrH Nov 11, 2024

PietrH Nov 11, 2024

PietrH Nov 11, 2024

PietrH Nov 11, 2024

PietrH Nov 11, 2024

PietrH commented Nov 11, 2024

	unique(purrr::pluck(deployments(x), "deploymentID")),
	unique(purrr::chuck(deployments(x), "deploymentID")),

	unique(purrr::pluck(deployments(y), "deploymentID"))
	unique(purrr::chuck(deployments(y), "deploymentID"))

		unique(purrr::pluck(media(x), "mediaID")),
		unique(purrr::pluck(media(y), "mediaID"))

		unique(purrr::pluck(observations(x), "observationID")),
		unique(purrr::pluck(observations(y), "observationID"))

		unique(purrr::pluck(media(x), "eventID")),
		unique(purrr::pluck(media(y), "eventID"))

New function merge_camtrapdp() #112

Are you sure you want to change the base?

New function merge_camtrapdp() #112

Conversation

sannegovaert commented Jul 25, 2024 • edited Loading

codecov bot commented Jul 25, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PietrH commented Nov 11, 2024

New function `merge_camtrapdp()` #112

New function `merge_camtrapdp()` #112

sannegovaert commented Jul 25, 2024 •

edited

Loading

codecov bot commented Jul 25, 2024 •

edited

Loading