Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Template: add option to exclude code linters for custom pipeline template #3084

Merged
merged 4 commits into from
Aug 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/create-test-lint-wf-template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ jobs:
- "template_skip_github_badges.yml"
- "template_skip_igenomes.yml"
- "template_skip_ci.yml"
- "template_skip_code_linters.yml"
runner:
# use the runner given by the input if it is dispatched manually, run on github if it is a rerun or on self-hosted by default
- ${{ github.event.inputs.runners || github.run_number > 1 && 'ubuntu-latest' || 'self-hosted' }}
Expand Down Expand Up @@ -102,6 +103,10 @@ jobs:
run: |
printf "org: my-prefix\nskip: nf_core_configs" > create-test-lint-wf/template_skip_nf_core_configs.yml
- name: Create template skip code_linters
run: |
printf "org: my-prefix\nskip: code_linters" > create-test-lint-wf/template_skip_code_linters.yml
# Create a pipeline from the template
- name: create a pipeline from the template ${{ matrix.TEMPLATE }}
run: |
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- Remove deprecated syntax ([#3046](https://github.com/nf-core/tools/pull/3046))
- Use filename in code block for `params.yml` ([#3055](https://github.com/nf-core/tools/pull/3055))
- Remove release announcement for non nf-core pipelines ([#3072](https://github.com/nf-core/tools/pull/3072))
- add option to exclude code linters for custom pipeline template ([#3084](https://github.com/nf-core/tools/pull/3084))

### Linting

Expand Down
3 changes: 2 additions & 1 deletion nf_core/pipeline-template/.gitpod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,9 @@ tasks:
vscode:
extensions: # based on nf-core.nf-core-extensionpack
#{%- if code_linters -%}
- esbenp.prettier-vscode # Markdown/CommonMark linting and style checking for Visual Studio Code
- EditorConfig.EditorConfig # override user/workspace settings with settings found in .editorconfig files
- EditorConfig.EditorConfig # override user/workspace settings with settings found in .editorconfig files{% endif %}
- Gruntfuggly.todo-tree # Display TODO and FIXME in a tree view in the activity bar
- mechatroner.rainbow-csv # Highlight columns in csv files in different colors
# - nextflow.nextflow # Nextflow syntax highlighting
Expand Down
8 changes: 8 additions & 0 deletions nf_core/pipelines/create/create.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,13 @@ def __init__(
".github/workflows/awstest.yml",
".github/workflows/release-announcements.yml",
],
"code_linters": [
".editorconfig",
".pre-commit-config.yaml",
".prettierignore",
".prettierrc.yml",
".github/workflows/fix-linting.yml",
],
}
# Get list of files we're skipping with the supplied skip keys
self.skip_paths = set(sp for k in skip_paths for sp in skippable_paths[k])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider the ability to include or exclude specific files via the .nf-core.yml here. Not clear how we should do this yet but it could be a specific key for the .nf-core.yml:

exclude:
  - .github/workflows/fix-linting.yml
  - .github/workflows/clean-up.yml
include:
  - .github/workflows/ci.yml

This would override the broad categories by the wizard. It could be bypassed when using the TUI for now, consider it an advanced feature.

Expand Down Expand Up @@ -202,6 +209,7 @@ def obtain_jinja_params_dict(self, features_to_skip, pipeline_dir):
"github_badges": {"file": False, "content": True},
"igenomes": {"file": True, "content": True},
"nf_core_configs": {"file": False, "content": True},
"code_linters": {"file": True, "content": True},
}

# Set the parameters for the jinja template
Expand Down
16 changes: 16 additions & 0 deletions nf_core/pipelines/create/custompipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,16 @@
For more information about nf-core configuration profiles, see the [nf-core/configs repository](https://github.com/nf-core/configs)
"""

markdown_code_linters = """
Pipelines include code linters to check the formatting of your code in order to harmonize code styles between developers.
Linters will check all non-ignored files, e.g., JSON, YAML, Nextlow or Python files in your repository.
The available code linters are:

- pre-commit (https://pre-commit.com/): used to run all code-linters on every PR and on ever commit if you run `pre-commit install` to install it in your local repository.
- editor-config (https://github.com/editorconfig-checker/editorconfig-checker): checks rules such as indentation or trailing spaces.
- prettier (https://github.com/prettier/prettier): enforces a consistent style (indentation, quoting, line length, etc).
"""


class CustomPipeline(Screen):
"""Select if the pipeline will use genomic data."""
Expand Down Expand Up @@ -80,6 +90,12 @@ def compose(self) -> ComposeResult:
"The pipeline will include configuration profiles containing custom parameters requried to run nf-core pipelines at different institutions",
"nf_core_configs",
),
PipelineFeature(
markdown_code_linters,
"Use code linters",
"The pipeline will include code linters and CI tests to lint your code: pre-commit, editor-config and prettier.",
"code_linters",
),
classes="features-container",
)
yield Center(
Expand Down
511 changes: 256 additions & 255 deletions tests/__snapshots__/test_create_app.ambr

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions tests/data/pipeline_create_template_skip.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ skip_features:
- github_badges
- igenomes
- nf_core_configs
- code_linters
13 changes: 7 additions & 6 deletions tests/test_create.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,14 +106,15 @@ def test_pipeline_creation_with_yml_skip(self, tmp_path):
pipeline.init_pipeline()

# Check pipeline template yml has been dumped to `.nf-core.yml` and matches input
assert not os.path.exists(os.path.join(pipeline.outdir, "pipeline_template.yml"))
assert os.path.exists(os.path.join(pipeline.outdir, ".nf-core.yml"))
with open(os.path.join(pipeline.outdir, ".nf-core.yml")) as fh:
assert not (pipeline.outdir / "pipeline_template.yml").exists()
assert (pipeline.outdir / ".nf-core.yml").exists()
with open(pipeline.outdir / ".nf-core.yml") as fh:
nfcore_yml = yaml.safe_load(fh)
assert "template" in nfcore_yml
assert yaml.safe_load(PIPELINE_TEMPLATE_YML.read_text()).items() <= nfcore_yml["template"].items()

# Check that some of the skipped files are not present
assert not os.path.exists(os.path.join(pipeline.outdir, "CODE_OF_CONDUCT.md"))
assert not os.path.exists(os.path.join(pipeline.outdir, ".github"))
assert not os.path.exists(os.path.join(pipeline.outdir, "conf", "igenomes.config"))
assert not (pipeline.outdir / "CODE_OF_CONDUCT.md").exists()
assert not (pipeline.outdir / ".github").exists()
assert not (pipeline.outdir / "conf" / "igenomes.config").exists()
assert not (pipeline.outdir / ".editorconfig").exists()
Loading