Skip to content

Commit

Permalink
Update/create task (#246)
Browse files Browse the repository at this point in the history
* update gitignore

* update notes on datsets

* fix file paths for API section

* minor fixes

* update changelog

---------

Co-authored-by: Michaela Mueller <[email protected]>
  • Loading branch information
rcannood and mumichae authored May 5, 2023
1 parent d959fd6 commit 92b3f19
Show file tree
Hide file tree
Showing 7 changed files with 36 additions and 33 deletions.
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
.DS_Store
/.quarto/
/_site
/_freeze
/.vscode
/.vscode
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# openproblems.bio v2.2.1

... nothing here yet ...
## BUG FIX

* Minor fixes to the documentation pages (#246).

# openproblems.bio v2.2.0

Expand Down
19 changes: 9 additions & 10 deletions documentation/create_task/design_api.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ In general, an API follows the structure of a viash component and can contain an
It is up you to decide which directives you consider general-purpose for your specific components.
An example could look like the following:

```{.yaml filename="api/component/generic_component.yaml"}
```{.yaml filename="src/<task_id>/api/component/generic_component.yaml"}
functionality:
namespace: <task>/<category> # <1>
namespace: <task_id>/<category> # <1>
info: # <2>
...
arguments: # <3>
Expand Down Expand Up @@ -64,16 +64,16 @@ cat src/label_projection/api/comp_method.yaml
```
````

### Connecting the component API to a specific component
### Applying the API to a specific component

A component can inherit from an API specification, which follows the basic viash component.
The directive for inheriting from an API specification is `__merge__` at the topmost level of the component API.
That way, all the specifications under the `functionality` directive of the API definition will be merged into the `functionality` component configuration.
These specifications do not need to be defined again in the component config, unless you want to override them for select tasks.
Note, that the API file path must be relative to the location of the `config.vsh.yaml` file.

```{.yaml filename="<task>/<category>/<subtask>/config.vsh.yaml"}
__merge__: ../../api/component/<>.yaml # <1>
```{.yaml filename="src/<task>/<category>/<component>/config.vsh.yaml"}
__merge__: ../../api/component/generic_component.yaml # <1>
functionality: # <2>
name: ...
description: ...
Expand All @@ -86,7 +86,7 @@ platforms: # <5>
```

1. The generic component type API specification file.
2. Specify any component-specific directives that are not in the generic API or overwrite directives if neeeded
2. Specify any component-specific directives that are not in the generic API or overwrite directives if needed.
3. The metadata of the component.
4. Resource files of the component (typically a Python or an R script).
5. Platform specification info (typically which Docker image to use and which Python/R packages to install on top of that).
Expand All @@ -100,7 +100,7 @@ While they can be defined in a viash config file or a component API directly, de
When specifying the API file formats, you should describe the contents, structure, and any additional metadata required for the files. This information will be useful for developers to understand the expected file formats and components quickly. Use YAML to define the file formats, as shown below:


```{.yaml filename="api/file/anndata_specification_1.yaml"}
```{.yaml filename="src/<task_id>/api/file/anndata_specification_1.yaml"}
type: file # <1>
description: ... # <2>
example: ... # <3>
Expand Down Expand Up @@ -148,10 +148,9 @@ cat src/label_projection/api/anndata_train.yaml
```
````

### Connecting the file API to a specific component
### Applying the API to a specific component

The File API YAML files are best stored under the `api/file` of a task in `src/<task>/`.
It can be used in a generic component API YAML or in the component configuration directly.
The File API YAML files can be used in a generic component API YAML or in the component configuration directly.
Note, that the API file path must be relative to the location of the corresponding component API or `config.vsh.yaml` file.

```{yaml}
Expand Down
6 changes: 1 addition & 5 deletions documentation/create_task/index.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,7 @@ description: Adding a new task from scratch
order: 30
---

This chapter provides a step-by-step overview of the process, including understanding the typical structure of an OpenProblems task, defining a new task, selecting datasets, designing the API for files and components, adding components, and creating a workflow that ties everything together.

Here are the main sections of this guide, each represented by a file with headers:
This chapter provides a step-by-step overview of the process of adding a new task. Here are the main sections:

1. **[Task Structure](task_structure.qmd)**: The structure of a typical OpenProblems task, including its components and file formats.

Expand All @@ -19,5 +17,3 @@ Here are the main sections of this guide, each represented by a file with header
5. **[Create Components](create_components.qmd)**: Dive into the process of adding components to your task, such as dataset loaders, methods, control methods, and metrics.

6. **[Create Workflow](create_workflow.qmd)**: Combine your components into a cohesive Nextflow workflow.

By following this guide, you'll gain a thorough understanding of the process and best practices for creating a new OpenProblems task from scratch.
14 changes: 8 additions & 6 deletions documentation/create_task/select_datasets.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,15 @@ This page is still under construction.
:::

:::{.content-hidden}
* Which fields are required
* Create split dataset component

workflow:

1. create API for processed datasets (starting from common data loader API)
2. create component for processing datasets
## Browse existing datasets

## Create a dataset processor

### Define the API
* Which fields are required

### Create the viash component
* Create split dataset component

:::
8 changes: 6 additions & 2 deletions documentation/fundamentals/index.qmd
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
---
title: Fundamentals
description: "..."
order: 10
---

...
This chapter introduces the main philosophy of the Openproblems project, key concepts, requirements, and a short getting started guide.

1. [Philosophy](philosophy.qmd): Introduces the main philosophy behind the OpenProblems project and its goals.
2. [Concepts](concepts.qmd): Covers key concepts, such as the AnnData file format and Viash components.
3. [Requirements](requirements.qmd): Provides a list of required software to start contributing to the OpenProblems project.
4. [Getting Started](getting_started.qmd): A step-by-step guide to set up your environment and start contributing to OpenProblems.
15 changes: 7 additions & 8 deletions documentation/index.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,21 @@ order: 1

Welcome to the OpenProblems documentation!

OpenProblems is a library for machine learning scientists, computational biologists, and single-cell data analysts to benchmark and compare single-cell omics methods on a variety of tasks. We believe that improving reproducibility and transparency in research will accelerate progress in single-cell omics.
OpenProblems is a framework for machine learning scientists, computational biologists, and single-cell data analysts to benchmark and compare single-cell omics methods on a variety of tasks. We believe that improving reproducibility and transparency in research will accelerate progress in single-cell omics.

This documentation provides an overview of the library and how to contribute to it. It is organized into several sections, including:
This documentation provides an overview of the framework and how to contribute to it. It is organized into several chapters, including:

* [Analyse results](analyze_results/): How to analyze the benchmarking results in Python and R.
* [Contribute](contribute/): How to contribute datasets, methods, or metrics to OpenProblems.
* [Create a new task](create_task/): How to create a new task.
* [Fundamentals](fundamentals/): the main philosophy of the Openproblems project, key concepts, requirements, and a short getting started guide.
* [Create components](create_component/): How to contribute datasets, methods, or metrics to OpenProblems.
* [Create a new task](create_task/): A step-by-step overview of the process of adding a new task
* [Reference](reference/): Reference documentation on the structure of the OpenProblems repositories.
* [More information](more_information/): Additional information such as our code of conduct, frequently asked questions, and troubleshooting tips.

We encourage everyone who uses OpenProblems to contribute to the library in any way they can. This can include submitting a pull request to improve the documentation, answering queries on the issue tracker, investigating bugs, reviewing other developers' pull requests, or simply reporting issues and giving a "thumbs up" to relevant issues.
We encourage everyone who uses OpenProblems to contribute to the framework in any way they can. This can include submitting a pull request to improve the documentation, answering queries on the issue tracker, investigating bugs, reviewing other developers' pull requests, or simply reporting issues and giving a "thumbs up" to relevant issues.

One simple but important way to contribute is to spread the word about the library by referencing it from your blog and articles, linking to it from your website, or starring it on GitHub to show that you use it. This not only helps us gain more visibility but also encourages others to try out the library and contribute to it.

[![](images/stargazers.png){width=500px}](https://github.com/openproblems-bio/openproblems-v2/stargazers)

To get started with contributing to the library, please see the "Contribute" section which provides information on the requirements, getting started, adding datasets, methods, and metrics, and running benchmarks. We also provide a detailed guide on how to create a new task in the "Create a new task" section.

Finally, we want to emphasize that OpenProblems is an inclusive community and we expect all members to adhere to our [code of conduct](more_information/code_of_conduct.qmd). We hope that this documentation helps you get started with OpenProblems and we look forward to your contributions.

0 comments on commit 92b3f19

Please sign in to comment.