Skip to content

WIP: POC Redesign extraction #256

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 117 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
117 commits
Select commit Hold shift + click to select a range
e0e1571
Add some initial definitions
llrs-roche Mar 3, 2025
707224e
Add some infrastructure
llrs-roche Mar 3, 2025
9f9aacb
Resolution tested up to variables
llrs-roche Mar 5, 2025
0d89dce
Add function for when a new selection is chosen
llrs-roche Mar 6, 2025
58182d2
Raising more errors and silencing some tests
llrs-roche Mar 7, 2025
c21591f
Improve update_spec
llrs-roche Mar 10, 2025
8023b1c
Simplify tests
llrs-roche Mar 10, 2025
37225be
Add print method
llrs-roche Mar 10, 2025
e1ec420
Uncomment some tests
llrs-roche Mar 10, 2025
a8bb226
Make print work
llrs-roche Mar 10, 2025
f4bf89a
Move delayed to attributes
llrs-roche Mar 12, 2025
a1678fa
Add missing method!
llrs-roche Mar 12, 2025
06c0eb9
Simplify type creation
llrs-roche Mar 12, 2025
208fb53
Simplify resolver
llrs-roche Mar 12, 2025
cb94d6d
Delayed takes into consideration the selection too
llrs-roche Mar 12, 2025
158eda9
Fix mistake and remove delayed class
llrs-roche Mar 12, 2025
d545331
Resolve better by names
llrs-roche Mar 12, 2025
99cadc2
Fix issues
llrs-roche Mar 12, 2025
b26695a
Simplify c.type
llrs-roche Mar 12, 2025
6600d17
Use Group Generic Functions
llrs-roche Mar 12, 2025
df47b48
Improve tests
llrs-roche Mar 12, 2025
e90025c
Make it work for simple cases of values
llrs-roche Mar 12, 2025
a92f051
Allow OR combinations
llrs-roche Mar 13, 2025
41800f3
Make it easier to work with types
llrs-roche Mar 17, 2025
3a39de2
Avoid conflicting names
llrs-roche Mar 17, 2025
6920d6e
Simplify according to new methods
llrs-roche Mar 17, 2025
d728739
Updating docs
llrs-roche Mar 17, 2025
6a85027
Fix some checks
llrs-roche Mar 17, 2025
9993468
Simplify and reorganize delayed
llrs-roche Mar 18, 2025
3624441
Basic operations work more flexible
llrs-roche Mar 19, 2025
941b172
Simplify code
llrs-roche Mar 25, 2025
fe138e9
Add more checks
llrs-roche Mar 25, 2025
cf2d634
Resolve only what is possible.
llrs-roche Mar 25, 2025
3385521
Combine safely multiple types
llrs-roche Mar 25, 2025
2c42af5
Add documentation for types
llrs-roche Mar 25, 2025
3cd9132
Better handling of types not resolved
llrs-roche Mar 25, 2025
b780107
Better handling of resolver
llrs-roche Mar 25, 2025
9cfb6f8
Better handling of composing
llrs-roche Mar 25, 2025
bad225d
Adding some other files
llrs-roche Mar 25, 2025
c4c8e40
Add update_spec
llrs-roche Mar 25, 2025
635ea9b
Module for simple specifications
llrs-roche Mar 25, 2025
a30ec58
Update for multiple selections
llrs-roche Mar 26, 2025
3031edc
Make sure a list of transformators work
llrs-roche Mar 26, 2025
81c3a9b
Allow lists (should be of transforms)
llrs-roche Mar 31, 2025
d7a005b
Remove some comments
llrs-roche Mar 31, 2025
fa89e05
Merging basic block
llrs-roche Mar 31, 2025
0142b78
[skip style] [skip vbump] Restyle files
github-actions[bot] Mar 31, 2025
521fd09
[skip roxygen] [skip vbump] Roxygen Man Pages Auto Update
github-actions[bot] Mar 31, 2025
54058d3
Rename to make it easier to read
llrs-roche Apr 2, 2025
957e59d
Minor tweaks
llrs-roche Apr 2, 2025
b440e3f
Merging for multiple data.frames
llrs-roche Apr 2, 2025
516c434
Add back the extract methods
llrs-roche Apr 2, 2025
e82f500
Add Method and the default (to be able to expand)
llrs-roche Apr 2, 2025
d4d826e
Extract multiple columns
llrs-roche Apr 4, 2025
56fa808
Improve naming and fix some bugs
llrs-roche Apr 4, 2025
bcbf304
Handle bare types
llrs-roche Apr 4, 2025
dabf731
Add some checks on the input
llrs-roche Apr 4, 2025
78a8e1f
Merging features
llrs-roche Apr 4, 2025
1426b06
Merge commit '521fd09460dc4eb57f21bd5b022cee59f311804a'
llrs-roche Apr 4, 2025
ee357b0
[skip style] [skip vbump] Restyle files
github-actions[bot] Apr 4, 2025
b266116
[skip roxygen] [skip vbump] Roxygen Man Pages Auto Update
github-actions[bot] Apr 4, 2025
8a2fcb6
Instead of a list make us of ...
llrs-roche Apr 7, 2025
b03e0d9
Solve most issues
llrs-roche Apr 7, 2025
7f14c03
Resolve conflicts
llrs-roche Apr 7, 2025
a19ed31
[skip style] [skip vbump] Restyle files
github-actions[bot] Apr 7, 2025
f146b4f
Fix some error/warnings
llrs-roche Apr 8, 2025
58fcf91
Apply styler
llrs-roche Apr 8, 2025
730aaa3
Merge branch 'redesign_extraction@main' of https://github.com/insight…
llrs-roche Apr 8, 2025
6aedd0a
[skip roxygen] [skip vbump] Roxygen Man Pages Auto Update
github-actions[bot] Apr 8, 2025
4807d7d
Add additional checks and case handling
llrs-roche Apr 9, 2025
33a18fc
Merge branch 'redesign_extraction@main' of https://github.com/insight…
llrs-roche Apr 9, 2025
711f92d
[skip roxygen] [skip vbump] Roxygen Man Pages Auto Update
github-actions[bot] Apr 9, 2025
06c4b07
Comment code related to MAE
llrs-roche Apr 9, 2025
8145d93
Merge branch 'redesign_extraction@main' of https://github.com/insight…
llrs-roche Apr 9, 2025
0d032cd
Minor improvements
llrs-roche Apr 9, 2025
c7f2675
Merge from a list of inputs
llrs-roche Apr 9, 2025
b733b34
Make it possible to consolidate input and extraction
llrs-roche Apr 10, 2025
1134bbd
Increase version
llrs-roche Apr 10, 2025
cc26a25
Merge data.frames within qenv
llrs-roche Apr 24, 2025
1c2ddad
WIP: add ! operator
llrs-roche Apr 24, 2025
4605b89
Merge branch 'main' into redesign_extraction@main
llrs-roche Apr 24, 2025
d27750f
[skip style] [skip vbump] Restyle files
github-actions[bot] Apr 24, 2025
810ce49
Address some checks warnings
llrs-roche Apr 24, 2025
8cea674
WIP: Add tests for ! operator
llrs-roche Apr 24, 2025
cf4c5b2
[skip style] [skip vbump] Restyle files
github-actions[bot] Apr 24, 2025
4a3c2c8
Use unique datasets before merging
llrs-roche Apr 25, 2025
59c6aff
Use consolidate_extraction to make sure merging is needed
llrs-roche Apr 25, 2025
11932a8
Verify it works without problems with just c()
llrs-roche Apr 25, 2025
7b306fc
[skip style] [skip vbump] Restyle files
github-actions[bot] Apr 25, 2025
bec03ff
Remove operators (and exclude)
llrs-roche Apr 28, 2025
3c32eab
Simplify
llrs-roche Apr 29, 2025
386232d
Add tidyselect::eval_select
llrs-roche Apr 29, 2025
64c42e2
Add support for expressions in types
llrs-roche Apr 30, 2025
2a00dd2
Use the tidyselect approach (remove operators)
llrs-roche Apr 30, 2025
a78ecce
Remove extra MAE methods
llrs-roche Apr 30, 2025
6fa687e
Fix issues
llrs-roche May 6, 2025
18df6fc
Fix remaining tests
llrs-roche May 6, 2025
5cadc43
Fix bugs
llrs-roche May 6, 2025
72c975f
Merge branch 'redesign_extraction@main' of https://github.com/insight…
llrs-roche May 6, 2025
50576d6
Add method for colData and avoid Infinite recursion
llrs-roche May 6, 2025
a031ad9
If selection is empty it stops
llrs-roche May 8, 2025
1535244
Rename x argument to names and transform class to specification
llrs-roche May 9, 2025
1385c1f
Revert testing to allow empty selection
llrs-roche May 9, 2025
85787b8
Fix problems with renamings
llrs-roche May 9, 2025
3fafc92
Extract before tidyselect on names
llrs-roche May 9, 2025
5ccfe34
Reuse extraction
llrs-roche May 9, 2025
10509da
Match expectations based on names
llrs-roche May 9, 2025
5b8b51d
Clean up of unused functions and arguments
llrs-roche May 9, 2025
29c6500
Restore test
llrs-roche May 9, 2025
3ba1bf5
Renaming
llrs-roche May 9, 2025
ced9053
Fix renaming issues
llrs-roche May 12, 2025
f3e3fa3
Remove unused package
llrs-roche May 12, 2025
55a8537
Renaming
llrs-roche May 12, 2025
6bf1af3
Select instead of merging
llrs-roche May 12, 2025
744a271
Add function to provide the list of inputs
llrs-roche May 12, 2025
c773304
Improve calls and merging
llrs-roche May 13, 2025
0638534
Fix unresolved updates
llrs-roche May 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ default_language_version:
python: python3
repos:
- repo: https://github.com/lorenzwalthert/precommit
rev: v0.4.3.9007
rev: v0.4.3.9009
hooks:
- id: style-files
name: Style code with `styler`
Expand Down
1 change: 1 addition & 0 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Imports:
shinyjs,
shinyvalidate (>= 0.1.3),
stats,
teal.code (>= 0.6.0),
teal.data (>= 0.7.0),
teal.logger (>= 0.3.1),
teal.widgets (>= 0.4.3),
Expand Down
31 changes: 31 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
@@ -1,12 +1,29 @@
# Generated by roxygen2: do not edit by hand

S3method(anyNA,type)
S3method(c,specification)
S3method(c,type)
S3method(data_extract_multiple_srv,FilteredData)
S3method(data_extract_multiple_srv,list)
S3method(data_extract_multiple_srv,reactive)
S3method(data_extract_srv,FilteredData)
S3method(data_extract_srv,list)
S3method(determine,colData)
S3method(determine,datasets)
S3method(determine,default)
S3method(determine,list)
S3method(determine,specification)
S3method(determine,values)
S3method(determine,variables)
S3method(extract,default)
S3method(extract,teal_data)
S3method(filter_spec_internal,default)
S3method(filter_spec_internal,delayed_data)
S3method(is.delayed,default)
S3method(is.delayed,list)
S3method(is.delayed,specification)
S3method(is.delayed,type)
S3method(is.na,type)
S3method(merge_expression_module,list)
S3method(merge_expression_module,reactive)
S3method(merge_expression_srv,list)
Expand All @@ -19,6 +36,7 @@ S3method(print,delayed_select_spec)
S3method(print,delayed_value_choices)
S3method(print,delayed_variable_choices)
S3method(print,filter_spec)
S3method(print,type)
S3method(resolve,default)
S3method(resolve,delayed_choices_selected)
S3method(resolve,delayed_data_extract_spec)
Expand All @@ -45,6 +63,9 @@ export(data_extract_spec)
export(data_extract_srv)
export(data_extract_ui)
export(datanames_input)
export(datasets)
export(determine)
export(extract)
export(filter_spec)
export(first_choice)
export(first_choices)
Expand All @@ -55,21 +76,31 @@ export(get_extract_datanames)
export(get_merge_call)
export(get_relabel_call)
export(is.choices_selected)
export(is.delayed)
export(is_single_dataset)
export(last_choice)
export(last_choices)
export(list_extract_spec)
export(mae_colData)
export(merge_datasets)
export(merge_expression_module)
export(merge_expression_srv)
export(module_input_server)
export(module_input_ui)
export(no_selected_as_NULL)
export(resolve_delayed)
export(resolver)
export(select_spec)
export(select_spec.default)
export(select_spec.delayed_data)
export(split_by_sep)
export(update_spec)
export(value_choices)
export(values)
export(variable_choices)
export(variables)
import(shiny)
importFrom(dplyr,"%>%")
importFrom(lifecycle,badge)
importFrom(methods,is)
importFrom(tidyselect,everything)
60 changes: 60 additions & 0 deletions R/delayed.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Only delay if the type or object really needs it and is not already delayed
delay <- function(x) {
if (is.delayed(x)) {
attr(x, "delayed") <- TRUE
}
x
}

#' Is the specification resolved?
#'
#' Check that the specification is resolved against a given data source.
#' @param specification Object to be evaluated.
#' @returns A single logical value.
#' @examples
#' is.delayed(1)
#' is.delayed(variables("df", "df"))
#' is.delayed(variables("df")) # Unknown selection
#' @export
is.delayed <- function(specification) {

Check warning on line 19 in R/delayed.R

View workflow job for this annotation

GitHub Actions / SuperLinter 🦸‍♀️ / Lint R code 🧶

file=R/delayed.R,line=19,col=1,[object_name_linter] Variable and function name style should match snake_case, symbols, ANL or ADaM.
UseMethod("is.delayed")
}

#' @export
#' @method is.delayed default
is.delayed.default <- function(specification) {
# FIXME: A warning?
FALSE
}

# Handling a list of transformers e1 | e2
#' @export
#' @method is.delayed list
is.delayed.list <- function(specification) {
any(vapply(specification, is.delayed, logical(1L)))
}

#' @export
#' @method is.delayed specification
is.delayed.specification <- function(specification) {
any(vapply(specification, is.delayed, logical(1L)))
}

#' @export
#' @method is.delayed type
is.delayed.type <- function(specification) {
if (!is.na(specification)) {
return(!all(is.character(specification$choices)) || !all(is.character(specification$selected)))
}
FALSE
}

resolved <- function(specification, type = is(specification)) {
s <- all(is.character(specification$choices)) && all(is.character(specification$selected))

if (!s && !all(specification$selected %in% specification$choices)) {
stop("Selected ", type, " not resolved.")
}
attr(specification, "delayed") <- NULL
specification
}
80 changes: 80 additions & 0 deletions R/extract.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
#' Internal method to extract data from different objects
#'
#' Required to resolve a specification into something usable (by comparing with the existing data).
#' Required by merging data based on a resolved specification.
#' @param x Object from which a subset/element is required.
#' @param variable Name of the element to be extracted.
#' @param ... Other arguments passed to the specific method.
#' @export
#' @examples
#' extract(iris, "Sepal.Length")
extract <- function(x, variable, ...) {
UseMethod("extract")
}


#' @export
extract.default <- function(x, variable, ..., drop = FALSE) {
if (length(dim(x)) == 2L || length(variable) > 1L) {
x[, variable, drop = drop]
} else {
x[[variable]]
}
}

#' @export
extract.teal_data <- function(x, variable, ...) {
if (length(variable) > 1L) {
x[variable]
} else {
x[[variable]]
}
}

# @export
# @method extract data.frame
# extract.data.frame <- function(x, variable) {
# # length(variable) == 1L

Check warning on line 37 in R/extract.R

View workflow job for this annotation

GitHub Actions / SuperLinter 🦸‍♀️ / Lint R code 🧶

file=R/extract.R,line=37,col=5,[commented_code_linter] Remove commented code.
# x[, variable, drop = TRUE]

Check warning on line 38 in R/extract.R

View workflow job for this annotation

GitHub Actions / SuperLinter 🦸‍♀️ / Lint R code 🧶

file=R/extract.R,line=38,col=5,[commented_code_linter] Remove commented code.
# }

# @export
# extract.qenv <- function(x, variable) {
# x[[variable]]

Check warning on line 43 in R/extract.R

View workflow job for this annotation

GitHub Actions / SuperLinter 🦸‍♀️ / Lint R code 🧶

file=R/extract.R,line=43,col=5,[commented_code_linter] Remove commented code.
# }

# Get code to be evaluated & displayed by modules
extract_srv <- function(id, input, data) {
stopifnot(is.null(input$datasets))
stopifnot(is.null(input$variables))
moduleServer(
id,
function(input, output, session) {
obj <- extract(data, input$datasets)
method <- paste0("extract.", class(obj))
method <- dynGet(method, ifnotfound = "extract.default", inherits = TRUE)
if (identical(method, "extract.default")) {
b <- get("extract.default")
} else {
b <- get(method)
}
# Extract definition
extract_f_def <- call("<-", x = as.name("extract"), value = b)
q <- teal.code::eval_code(data, code = extract_f_def)

# Extraction happening:
# FIXME assumes only to variables used
output <- call("<-",
x = as.name(input$datasets), value =
substitute(
extract(obj, variables),
list(
obj = as.name(input$datasets),
variables = input$variables
)
)
)
q <- teal.code::eval_code(q, code = output)
}
)
}
Loading
Loading