teal Addin - allow users to pick modules, based on which default app templates and project structure will be created

[kumamiao]

Please see details on the linked draft proposal below

Draft proposal

Issues:

• Research rhino for teal skeleton  #58
• Research Rstudio tools for teal skeleton #59
• Solution for modules-marketplace #51
• Research Github Template for teal skeleton #66
• Research cookiecutter for a teal app template #67
• Make teal.marketplace #53
• Filter and search feature for the modules marketplace #52
• Make button to get skelleton #54
• Make UI prototype for teal_marketet_module #55
• Improve UI for modules selection #56

Later we can improve:

• arguments customization
• data customization

Note: Be aware that app.R might grow too big, consider splitting into several files sourced in app.R (data.R, args.R, etc.)

[Polkas]

We should consider an update of our template for NEST apps, to take advantage from last shiny/rstudio improvements.
Here we could find 3 different templates (3 branches) https://github.com/rstudio/shiny-testing-gha-example/tree/minimal for shiny app.

This is mainly a research task.
The goal is to assess the value added for the end user, more precisely if this RStudio proposition could be helpful.
After the research an update could be done.
Definitely part of this item assume we are already open source.

Important topics to explore:

• template button functionality - USE THIS TEMPLATE (button)
  
• tests dir structure
• R dir automatically consumed, when run and deploy the app

[pawelru]

Few open-source initiatives that are very close to what we want to achieve:

• https://github.com/ThinkR-open/golem
• https://github.com/Appsilon/rhino
• https://github.com/devOpifex/leprechaun

[gogonzo]

Issue for a design

[kpagacz]

Here's my doc for specifying functional requirements: https://docs.google.com/document/d/1QvHhGiC94mx2xJ7pTEmGF14_YRbKiZ0eM1xpbQC0Ifg/edit

Some draw.io: https://drive.google.com/file/d/1nwxnceqxTLhosX4deLfvqd9RuYO75yxz/view?usp=sharing

[kpagacz]

https://www.figma.com/file/LZTe9lIfuQbDAwd9NuZSPL/TealDesigner?node-id=0%3A1

[Polkas]

https://app.diagrams.net/#G1P904WifNz_HnyTZyjUxYRYnbOBbJYBIT

https://docs.google.com/document/d/1Vys4wmUtAsbuUG84Y76oSWX2Ie4KwrauBktk-ju3X1Y/edit

[kpagacz]

Here are my notes from the meeting.

Dawid: 
* had a comment about scraping parameter descriptions from the Roxygen descriptions and showing them in the GUI of the designer


Pawel:
* wanted to open for modules not only from NEST
* wanted to decouple arguments specification from the rest of NEST, specifically the data extract specs and similar (Konrad: I do not know how feasible it is)
* specifying arguments is too much - do not create them at all

People seem to like the idea of module listings

Maciej:

tmc_n <- asNamespace("teal.modules.clinical")

lapply(ls(tmc_n)[grep("tm_", ls(tmc_n))], function(x) tmc_n[[x]])
modules_names <- ls(tmc_n)[grep("tm_", ls(tmc_n))]
tm_formals <- lapply(modules_names, function(x) formals(tmc_n[[x]]))
names(tm_formals) <- modules_names
tm_formals

and possibly docs/roxygen2 processing

[kpagacz]

I am closing this one.

[nikolas-burkoff]

Hi all @kumamiao @gogonzo @pawelru @telepath37 @npaszty @dinakar29

I've had an interesting discussion with a colleague about this and want to share some thoughts.

In my mind there is a clear difference between "teal.skeleton" which creates a simple project skeleton file structure/folder structure to write your apps in and "teal.designer" which does much more, creating modules, filling in arguments, adding in data etc. etc. I think teal.skeleton is definitely a good idea what follows below is about "teal.designer".

There is definitely value in adding configuration scripts and then UIs - think of the nice example of provisioning services for say AWS - you can do everything in the command line but that quickly becomes painful, so then there is an api layer so you can make scripts to do it, but then you start to always want to do similar things "provision me x servers" at which point you don't need a long script, but you write a configuration file which then generates the script which calls the api. Then after this you get to an even higher level "I want an AWS sagemaker instance" (possibly triggered by pressing some UI component) - this then creates the provisioning script which generates the configuration file which eventually calls the api to get what you need.

The key point though is higher levels should be more specialist I cannot do everything I can in the console at the higher levels - if I need that extra functionality, I drop down a level to do so. The more common/important a use case is, the higher level I can use in general - this has the benefit of shielding people from lower levels i.e. "create me AWS sagemaker instance button" can remain the same even if the underlying provisioning scripts change.

How does this relate to teal.designer -> if we reproduce most/all the functionality of the teal app in a UI then we are not becoming more specialist - we have kept the same problem of making an app but made it with a UI (sure you don't need to know the syntax of how to create a "data extract spec" but you still need to understand what all the options are and what they mean in order to make the app you want).

So is there a way of making more specific designers? I think there is - are there standard types of apps people want to generate? Yes - see the sample apps - and maybe other standard use cases I'm not aware of - so maybe we create a designer which allows people to generate a version of a sample app (with the code in appropriate files etc.) - so taking the efficacy app as an example things which need to be configured for that app should be able to be configured easily in that designer and you can then very quickly get an app which does what you want - which importantly can be added to (dropping down a level) if needed.

Similarly if 95% of cases of the efficiacy app use rice to pull in the data then we can in that app have configuration options (i.e. rice paths) to pull in data rather than opening the designer up to pull in whatever data from whatever connector (and in the 5% of cases which do not use rice there is an option to "configure data later")

This sort of thing may overlap with/be part of chevron as well so some more analysis may be required here but to me this seems more valuable than something general.

Finally it probably makes sense to have a configuration file between any UI and the generation of the code (e.g. a configuration yaml) - and a way of converting that into a full app script. There are many reasons for this 1) It's much easier to throw a UI over the top of a config file 2) It's easier to reason about and make sure we have everything we need in a script 3) easier to test 4) the UI is then language agnostic 5) we may actually decide that we don't need a UI if it's easy enough to write a config file

[cicdguy]

• @donyunardi ☝🏽 FYI

[Polkas]

@nikolas-burkoff thank you for this great view on the problem.

Summing up most important parts are:

• understand the true objective - "teal.skeleton"  vs "teal.designer"
• without making teal.designer more specialist then we have kept the same problem of making an app but made it with a UI.
• middleware configuration yaml files
• reusage of the sample apps which should covers 90% of use cases

Please read the original post.

[donyunardi]

I had a conversation with @kumamiao recently and we don't need to include the ability to specify the arguments for the selected modules in the addin.

This means, it is most likely that the code provided from the addin will not be functional and that is okay. The hope is that the code provided can serve as a starting point to accelerate users' teal learning experience as they tweak the starting code and try to build a functional teal app.

[donyunardi]

Sanofi did a great presentation on this during PHUSE US Connect 2024.
The name of the product is teal.builder.

[lcd2yyz]

Links to Sanofi' presentation and paper write up