Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept functions #1393

Open
wants to merge 23 commits into
base: main
Choose a base branch
from
Open

Accept functions #1393

wants to merge 23 commits into from

Conversation

gogonzo
Copy link
Contributor

@gogonzo gogonzo commented Oct 22, 2024

Closes #1352

This PR enables including any data type in the data (teal_data) object.

  • unfilterable datasets (not data.frame nor MAE) are not included in the filter-panel, but they are preserved in the data
  • unsupported data types are displayed in the data-summary-table but they are hidden by default
  • if any unsupported dataset is in the data they data-summary displays "show/hide unsupported" to toggle rows containing unsupported
  • functions are excluded from a hash calculation and this code is not included in SRC
hide unsupported show unsupported
image image
App example
devtools::load_all("teal.slice")
devtools::load_all("teal")
options("teal.bs_theme" = bslib::bs_theme(version = "5"))

data <- teal_data() |>
  within({
    library(MultiAssayExperiment)
    data(miniACC, envir = environment())
    iris <- iris
    foo <- function(x) cat("hello\n")
    vector <- letters
  })

modules <- modules(
  example_module(
    transformers = teal_transform_module(server = function(id, data) {
      moduleServer(id, function(input, output, session) {
        reactive({
          data() |> within({
            foo2 <- function() NULL
          })
        })
      })
    })
  ),
  example_module(datanames = "iris")
)

app <- init(data = data, modules = modules)

runApp(app)

Allow functions and enable to extend summary table
- add/modify tests
- r cmd check
@gogonzo gogonzo added the core label Oct 22, 2024
@gogonzo gogonzo changed the title 1352 accept function@main Accept functions Oct 22, 2024
Copy link
Contributor

github-actions bot commented Oct 22, 2024

Unit Tests Summary

  1 files   25 suites   8m 55s ⏱️
265 tests 261 ✅ 4 💤 0 ❌
519 runs  515 ✅ 4 💤 0 ❌

Results for commit 8630957.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Oct 22, 2024

Unit Test Performance Difference

Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
module_teal 💔 $63.46$ $+4.27$ $-4$ $0$ $0$ $0$
shinytest2-data_summary 💔 $37.12$ $+13.36$ $+1$ $0$ $0$ $0$
shinytest2-landing_popup 💔 $45.20$ $+1.53$ $0$ $0$ $0$ $0$
shinytest2-teal_data_module 💚 $48.76$ $-1.10$ $0$ $0$ $0$ $0$
shinytest2-teal_slices 💚 $61.74$ $-1.24$ $0$ $0$ $0$ $0$
Additional test case details
Test Suite $Status$ Time on main $±Time$ Test Case
module_teal 👶 $+0.49$ displays_unsupported_datasets
module_teal 👶 $+5.71$ summary_table_displays_MAE_dataset_added_in_transforms
shinytest2-data_summary 💀 $8.62$ $-8.62$ e2e_data_summary_displays_datasets_by_datanames_order_if_no_join_keys
shinytest2-data_summary 💀 $3.64$ $-3.64$ e2e_data_summary_displays_datasets_by_topological_sort_of_join_keys
shinytest2-data_summary 💀 $13.96$ $-13.96$ e2e_data_summary_is_displayed_properly_if_teal_data_include_data.frames_with_join_keys_MAE_objects_and_vectors
shinytest2-data_summary 💀 $3.42$ $-3.42$ e2e_data_summary_is_displayed_with_2_columns_data_without_keys
shinytest2-data_summary 💀 $3.56$ $-3.56$ e2e_data_summary_is_displayed_with_3_columns_for_data_with_join_keys
shinytest2-data_summary 👶 $+8.39$ e2e_data_summary_just_list_the_unfilterable_objects_at_the_bottom_when_provided
shinytest2-data_summary 💀 $3.91$ $-3.91$ e2e_data_summary_list_only_data_names_if_there_is_no_MAE_or_data.frames_in_teal_data
shinytest2-data_summary 👶 $+8.75$ e2e_data_summary_table_displays_datasets_by_datanames_order_if_no_join_keys
shinytest2-data_summary 👶 $+8.12$ e2e_data_summary_table_displays_datasets_by_topological_sort_of_join_keys
shinytest2-data_summary 👶 $+9.29$ e2e_data_summary_table_does_not_list_unsupported_objects
shinytest2-data_summary 👶 $+7.90$ e2e_data_summary_table_is_displayed_with_2_columns_data_without_keys
shinytest2-data_summary 👶 $+8.02$ e2e_data_summary_table_is_displayed_with_3_columns_for_data_with_join_keys

Results for commit 17da5e7

♻️ This comment has been updated with latest results.

- don't export get_filter_overview
Copy link
Contributor

github-actions bot commented Oct 22, 2024

badge

Code Coverage Summary

Filename                          Stmts    Miss  Cover    Missing
------------------------------  -------  ------  -------  ----------------------------------------------------------------------------------------------------------------------------------------
R/checkmate.R                        24       0  100.00%
R/dummy_functions.R                  47      11  76.60%   27, 29, 41, 52-59
R/get_rcode_utils.R                  12       0  100.00%
R/include_css_js.R                   22      17  22.73%   12-38, 76-82
R/init.R                            108      50  53.70%   107-114, 160-169, 171, 183-204, 229-232, 239-245, 248-249, 251
R/landing_popup_module.R             25      25  0.00%    61-87
R/module_bookmark_manager.R         158     127  19.62%   47-68, 88-138, 143-144, 156, 203, 238-315
R/module_data_summary.R             201      38  81.09%   26-54, 78, 216, 221, 235, 266-270
R/module_filter_data.R               64       2  96.88%   22-23
R/module_filter_manager.R           230      57  75.22%   56-62, 73-82, 90-95, 108-112, 117-118, 291-314, 340, 367, 379, 386-387
R/module_init_data.R                 74       0  100.00%
R/module_nested_tabs.R              236      91  61.44%   40-142, 174, 199-201, 320, 360
R/module_snapshot_manager.R         216     146  32.41%   89-95, 104-113, 121-133, 152-153, 170-180, 184-199, 201-208, 215-230, 234-238, 240-246, 249-262, 265-273, 304-318, 321-332, 335-341, 355
R/module_teal_data.R                149      10  93.29%   41-48, 84, 135-136
R/module_teal_lockfile.R            131      44  66.41%   32-36, 44-56, 59-61, 75, 85-87, 99-101, 109-118, 121, 123, 125-126, 160-161
R/module_teal_with_splash.R          12      12  0.00%    22-38
R/module_teal.R                     190      87  54.21%   48-143, 158, 184-185, 217
R/module_transform_data.R            54      32  40.74%   17-52
R/modules.R                         181      32  82.32%   173-177, 232-235, 333-334, 366-380, 418, 430-438
R/reporter_previewer_module.R        19       2  89.47%   30, 34
R/show_rcode_modal.R                 24      24  0.00%    17-42
R/tdata.R                            14      14  0.00%    19-61
R/teal_data_module-eval_code.R       24       0  100.00%
R/teal_data_module-within.R           7       0  100.00%
R/teal_data_module.R                 58       0  100.00%
R/teal_data_utils.R                  32       0  100.00%
R/teal_reporter.R                    68       6  91.18%   69, 77, 125-126, 129, 146
R/teal_slices-store.R                29       0  100.00%
R/teal_slices.R                      63       0  100.00%
R/TealAppDriver.R                   353     353  0.00%    52-735
R/utils.R                           230      37  83.91%   385-433
R/validate_inputs.R                  32       0  100.00%
R/validations.R                      58      37  36.21%   110-377
R/zzz.R                              15      11  26.67%   4-18
TOTAL                              3160    1265  59.97%

Diff against main

Filename                   Stmts    Miss  Cover
-----------------------  -------  ------  -------
R/module_data_summary.R      +12     -30  +17.07%
R/utils.R                    +48     +37  -16.09%
TOTAL                        +60      +7  +0.55%

Results for commit: 8630957

Minimum allowed coverage is 80%

♻️ This comment has been updated with latest results

@pawelru
Copy link
Contributor

pawelru commented Oct 23, 2024

this topic is very close to insightsengineering/teal.slice#621 issue. Would it make sense to include it or you prefer to do this separately?

@gogonzo
Copy link
Contributor Author

gogonzo commented Oct 23, 2024

this topic is very close to insightsengineering/teal.slice#621 issue. Would it make sense to include it or you prefer to do this separately?

Nice, I guess this solves the problem, is it?

@pawelru
Copy link
Contributor

pawelru commented Oct 23, 2024

this topic is very close to insightsengineering/teal.slice#621 issue. Would it make sense to include it or you prefer to do this separately?

Nice, I guess this solves the problem, is it?

I haven't tried it myself but based on below it seems that yes!

unsupported data types are displayed in the data-summary-table but they are hidden by default

@@ -3,15 +3,17 @@
#' Module and its utils to display the number of rows and subjects in the filtered and unfiltered data.
#'
#' @details Handling different data classes:
#' `get_object_filter_overview()` is a pseudo S3 method which has variants for:
#' `get_filter_overview()` is a pseudo S3 method which has variants for:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future it can be easily changed into S3 - so that app developers/teal-developers can make own methods to display summary for other datasets. There is a vision for teal.slice that it could be extended in the future by S3 methods.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make it S3 without exporting the method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can but I wouldn't do this now. This won't even change the code dramatically because we can't export <method>.data.frame or <method>.MultiAssayExperiment because it will complicate overriding in the future by the users. When trying to refactor teal.slice (you were also involved i remember), we noticed that overriding a S3 method is possible only through .GlobalEnv - if somebody creates <method>.data.frame in the package then teal will always prioritize teal method. This is why I'd avoid creating S3 for now, because the only method will be get_filter_overview.default with this "ugly-if--pseudo-dispatch"

R/module_data_summary.R Outdated Show resolved Hide resolved
dataname = dataname,
subject_keys = subject_keys
)
}
)

unssuported_idx <- vapply(rows, function(x) all(is.na(x[-1])), logical(1)) # this is mainly for vectors
do.call(rbind, c(rows[!unssuported_idx], rows[unssuported_idx]))
do.call(.smart_rbind, out)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.smart_rbind because get_filter_overview output is not restricted. Previously every method have returned:

  • dataname,
  • obs and obs_filtered: number of filtered and unfiltered rows which were formatted into single column in renderUI to Obs = <filtered>/<unfiltered>
  • subjects and subjects_filtered

Now methods can return any columns but formatting occurs in the method and not in the renderUI

  • dataname
  • obs: formatted counts "<filtered>/<unfiltered>"
  • subjects

I've modified the way what function returns to allow easy extension in the future. .rbind_smart is used only to bind overviews which might be different for different object types.

@donyunardi donyunardi assigned averissimo and unassigned vedhav Oct 28, 2024
Copy link
Contributor

@averissimo averissimo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the idea and the implementation makes sense, however, 🔴 the functionality is not working on my machine as the datanames are always showing (by default) and JS link does nothing

Screencast.from.2024-10-30.08-28-12.webm

Can you confirm it works to see if I need to research locally why my laptop is acting up 😅

Note: there's a conflict with main, but I suppose it's a simple fix to include the .smart_rbind function definition


I've added a few comments and found a bug in data summary (not this functionality, but I believe we can sneak the fix here).

I have 1 suggestion about the feature:

  • Group together unsupported data types on bottom

My sample app to test a bunch of unsupported types.

Example app
pkgload::load_all("teal")

options(
  teal.log_level = "TRACE",
  teal.show_js_log = TRUE,
  # teal.bs_theme = bslib::bs_theme(version = 5),
  shiny.bookmarkStore = "server"
)

init(
  data = teal_data() |> within({
    my_fun <- function(x) x
    `%cbind%` <- iris
    my_expression <- expression(1+1)
    my_primitive <- c
    my_env <- new.env()
    my_env$a <- 1
    my_env$b <- 2
    my_lang <- quote(1+1)
    my_expr <- expression(1+1)
    my_quo <- rlang::quo(1+1)
    my_numeric1 <- 2

    my_order0 <- 1
    my_order1 <- iris
    my_order2 <- 1

    iris <- iris
    mae <- withr::with_package("MultiAssayExperiment", {
      data("miniACC", package = "MultiAssayExperiment", envir = environment())
      miniACC
    })
  }),
  modules = modules(
    example_module("Module (mae unsupported)", datanames = c("mae")),
    example_module("Module (small)", datanames = c("iris", "my_order0", "my_order1", "my_order2")),
    example_module("Module (all)"),
    example_module(
      "With Transformers",
      transformers = list(
        teal_transform_module(
          server = function(id, data) {
            reactive({
              data() |> within({
                t_small_iris <- head(iris)
                t_character <- "transformed"
                t_mae <- withr::with_package("MultiAssayExperiment", {
                  data("miniACC", package = "MultiAssayExperiment", envir = environment())
                  miniACC
                })
              })
            })
          }
        )
      )
    )
  )
) |> shiny::runApp()

R/module_data_summary.R Outdated Show resolved Hide resolved
R/utils.R Outdated Show resolved Hide resolved
R/utils.R Show resolved Hide resolved
R/module_data_summary.R Outdated Show resolved Hide resolved
@@ -3,15 +3,17 @@
#' Module and its utils to display the number of rows and subjects in the filtered and unfiltered data.
#'
#' @details Handling different data classes:
#' `get_object_filter_overview()` is a pseudo S3 method which has variants for:
#' `get_filter_overview()` is a pseudo S3 method which has variants for:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make it S3 without exporting the method?

R/module_data_summary.R Outdated Show resolved Hide resolved
R/module_data_summary.R Outdated Show resolved Hide resolved
DESCRIPTION Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Dawid. This is very nicely implemented now and heading into being merged.

That always makes me step back and try again to look from a bigger perspective. I do understand the need of presenting the information which objects do not end up being filterable. Is this the right way to communicate this? I have some doubts.
I'm having this issue in my mind which is actually coming from our examples (most likely in teal.gallery). Have a look how big is the empty space. And space on the filter panel is really precious. You have added a show/hide for this. Is this the best solution? Me, as a user, I just want to know that a, b, c objects are not supported. For me it's fine to put that as a table footer (or text below) and enumerate names by comma. Adding this to the table unnecessarily (!) consumes a lot (!) of space.
I also have concerns about show/hide. This is a summary table and that suppose to be a simple information provided in a simple way. To me, It feels like an unnecessary feature that distracts from the main focus (that should be putted elsewhere - filtering, encoding, main panel).

I'm sorry to put my comment that late but I want you to have a second thought about this before it got merged.

(even though it's a general comment I'm putting this a code-comment to better organise the discussion)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries. Makes sense to just enumerate unsupported datanames. What form are you proposing? Like this:

Dataset Obs
iris 150/150
mtcars 27/27

Unsupported: a, b, potentially long, elo, metoo


I'm cool with reducing complexity and no interactivity with show/hide unsupported names 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. That was my first thought. We just need to think about HTML formatting - proper font size. Also, what to do when the string would be super wide. Maybe horizontal scrolling?

Yet another idea (probably worse) could be to enumerate in the tooltip - something like "hoover on me to see unsupported obj names)

Speaking about tooltip - maybe an additional info why on the "?" icon.
"Unsupported: . (?)" and in the tooltip on "(?)" -> "These objects are of class that is currently not supported in the Filter Panel"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Me personally like the tooltip idea more. Same tooltip which is now for the unsupported in the summary table - warning sign, when moving cursor over extra info is shown

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright. Let's try with this then

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do the names really matter that much to the user? If the list is long enough then it could take 2 or more lines

Alternatively, we we just have those in the tooltip

image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That also looks good 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately circle without filling is in font-awesome Pro. I've added following:

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open for suggestions here. I just wanted to highlight that unsupported could be too strong as these datasets just can't be filtered. They still can be transformed and used in the module as normal.


My suggestion for future is to have these counts as S3 method which can be extended quickly by us or app developer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just wanted to highlight that unsupported could be too strong as these datasets just can't be filtered.

I agree. Good move

From my side:

  • "dataset(s)" -> "object(s)"
  • icon to the right after the text
  • "+ X objs" -> "And X more objs"
  • smaller font / spacing?

name = "fas fa-circle-info",
title = paste(
sep = "",
collapse = ", ",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
collapse = ", ",
collapse = "\n",

I'd prefer having a line break instead of comma separation.
Screenshot 2024-11-01 153522

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

")"
)
),
sprintf("+%s unfilterable dataset(s)", sum(is_unsupported))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
sprintf("+%s unfilterable dataset(s)", sum(is_unsupported))
sprintf("+%s unfilterable object(s)", sum(is_unsupported))

Since we include non-dataset objects here I liked the previous naming where we called it just object.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: functions in data raise warnings
5 participants