Skip to content

Commit

Permalink
Transfer functional parts over
Browse files Browse the repository at this point in the history
  • Loading branch information
TCLamnidis committed Aug 28, 2023
1 parent 35dc826 commit c48445a
Show file tree
Hide file tree
Showing 12 changed files with 868 additions and 0 deletions.
5 changes: 5 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
blank_issues_enabled: false
contact_links:
- name: "Join the poseidon Slack space"
url: https://poseidon-8el7276.slack.com/
about: Discussions about everything Poseidon-Framework.
51 changes: 51 additions & 0 deletions .github/ISSUE_TEMPLATE/request_a_package.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
name: "Please add: [PACKAGE_NAME]"
about: Request to add data from a publication to the Poseidon Package Directory
labels: "new package"
---

<!--
# poseidon-framework/minotaur-recipes package request
Hello there!
Thanks for suggesting a new publication to add to the Poseidon Package Directory!
Please ensure you are completing all the TODOs outlined in these comments for each section.
-->

## Check for existing package

I have checked the following places for the requested package and could not find
it:

- [ ] Poseidon Package Directory
- [ ] The `packages/` directory in this repository
- [ ] Open issues in this repository
- [ ] Open pull requests in this repository

<!--
TODO: Please confirm that you checked the resources listed above to ensure that the requested package does not already exist under a different name than you expected.
-->

## Required information for the package

- DOI of publication: https://doi.org/10.1038/s41467-018-07483-5 <- Please
replace this example DOI with the one for the relevant publication
- Project accession number:
[PRJEB29360](https://www.ebi.ac.uk/ena/data/view/PRJEB29360) <- Please replace
this example ENA project link with the one for the relevant publication

<!--
TODO: Please replace the DOI and the study accession (and URL to the public sequencing data repository) above with the relevant information for the requested package
-->

## Will you work on this?

If you are planning to work on comppiling the `.ssf` file required to add this
package, please assign yourself to this issue. This helps the community
collaborate effectively and avoid duplicating work.

<!--
We really appreciate users compiling the SSF file for packages they want added to the Poseidon Package Directory.
If you would like to contribute but are unsure how, feel free to ask the community for help/advice at: https://poseidon-8el7276.slack.com
-->
54 changes: 54 additions & 0 deletions .github/ISSUE_TEMPLATE/update_a_package.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
name: "Please update: [PACKAGE_NAME]"
about: Suggest an update to an already-existing poseidon package.
labels: "package update"
---

<!--
# poseidon-framework/minotaur-recipes package request
Hello there!
Thank you for suggesting changes to a package in the Poseidon Package Directory!
Please ensure you are completing all the TODOs outlined in these comments for each section.
-->

## Which package needs changes?

Package name: `2018_Lamnidis_Fennoscandia` <- Replace the package name here with
the name of the relevant package.

<!-- TODO: Please Specify the name of the package you think needs updating -->

## Check for upcoming changes

I confirm there are no changes already suggested for this package that address
my issue in:

- [ ] [Open Pull Requests](https://github.com/poseidon-framework/minotaur-recipes/pulls)
- [ ] [Open Issues](https://github.com/poseidon-framework/minotaur-recipes/issues)

<!--
TODO: Please confirm you have looked for open issues and/or pull requests suggesting changes to this package and found none.
If an open issue or pull request exists already, and it might be addressing your issue, then consider mentioning your problem there instead of creating a new issue.
-->

## What needs changing, and why?

<!--
TODO: Please describe what you would like to see changed in the poseidon package.
Are there some data/individuals missing? Do you disagree with the nf-core/parameters used? The more information you provide here, the better.
-->

## (optional) What are the changes you'd like?

<!--
TODO: If you have a specific parameter that you would like to see changed, please list it here. Otherwise you can leave this section empty.
-->

## (optional) How did you realise something needs updating?

<!--
TODO: If you came across an error with the Poseidon package resulting from this processing while running some analysis, please give information that might help us reproduce the problems here.
If this section is not relevant, you can leave it empty.
-->
51 changes: 51 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
<!--
# poseidon-framework/minotaur-recipes package request
Hello there!
Thanks for suggesting a new publication to add to the Poseidon Package Directory!
Please ensure you are completing all the TODOs outlined in these comments for each section.
-->

Closes #XXX <!-- TODO: Please link the issue requesting the package here. -->

## PR Checklist

- [ ] Add the appropriate label to your PR (`new package` or `package_update`).
- [ ] The PR title is in the format `Add/update package: {package_name}`.
- [ ] The PR description includes a link to the issue requesting the package its update. (Add to `Closes #XXX` above.)

If adding a package:

- [ ] This PR contains a sequencingSourceFile (`.ssf`) for the requested package.
- [ ] The name of the `.ssf` file(s) matches the package name (i.e. `packages/2023_my_package/2023_my_package.ssf`).
- [ ] Comment `@delphis-bot create backbone` to this pull request to awaken Poseidon's trusty helper. (This should be repeated whenever changes are made to the SSF file contents).

Delphis-bot will then add a number of files to the PR. Check that they are all there:

- [ ] The file `packages/{package_name}/{package_name}.tsv` was added to the PR.
- [ ] The file `packages/{package_name}/{package_name}.tsv_patch.sh` was added to the PR from template.
- [ ] The file `packages/{package_name}/script_versions.txt` was added to the PR.
- [ ] The file `packages/{package_name}/{package_name}.config` was added to the PR from template.
<!-- TODO: Follow the steps outlined above and tick them off as you go. -->

## Human validation

<!-- TODO: Please do the minimal validation of the files outlined below -->

### Package SSF file (`*.ssf`)

- [ ] I confirm that the `poseidon_IDs`, `udg`, and `library_built` are filled and correct.
- [ ] I made sure to leave `notes` where necessary to explain any special cases/judgement calls made for data entries.

### Package TSV file (`*.tsv`)

- [ ] I confirm that the `UDG`, `Strandedness` columns are correct for each library.
- [ ] I confirm that the `R1_target_file` and `R2_target_file` columns point to the correct FastQ files for the library (i.e. consistent with SSF file).

### Package config file (`*.config`)

The template config file includes a few `TODO` statements, and information about them. Please ensure that you:

- [ ] I have selected the appropriate config for the CaptureType of the package.
- [ ] If any nf-core/eager parameters need to be altered from their defaults, I have added them within the `params` section at the end of the package config file.
47 changes: 47 additions & 0 deletions .github/workflows/PR_checks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: CI for PRs that add or update packages
# This workflow runs minimal required validation steps for merging of PRs that add or update packages
on:
pull_request:
branches:
- main

jobs:
ssf_validation:
name: Validate SSF
## Run on PRs that contain the right label
if: contains(github.event.pull_request.labels.*.name, 'new package') || contains(github.event.pull_request.labels.*.name, 'package update')
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0

## Commands that require a github token cannot be used here, because forked repos do not have access to secrets.
## Use the gh cli to check out the PR
# - name: Checkout Pull Request
# run: gh pr checkout ${{ github.event.pull_request.number }}
# env:
# GITHUB_TOKEN: ${{ secrets.delphis_bot_auth_token }}

- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: 3.10.11

- name: Get changed files
id: changes
# Set outputs using the command.
run: |
echo "PR base sha: ${{ github.event.pull_request.base.sha }}"
echo "PR head sha: ${{ github.sha }}"
echo "ssf=$(git diff --name-only --diff-filter=ACMRT ${{ github.event.pull_request.base.sha }} ${{ github.sha }} | grep '.ssf$')" >> $GITHUB_OUTPUT
- name: Check SSF
if: ${{ steps.changes.outputs.ssf }}
run: |
## Validate contents of each SSF file.
for ssf_fn in ${{ steps.changes.outputs.ssf }}; do
echo "Validating: '${ssf_fn}'"
python scripts/ssf_validator.py ${ssf_fn}
done
111 changes: 111 additions & 0 deletions .github/workflows/create_package_skeleton.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: Create additional PR files
on:
issue_comment:
types: [created]

jobs:
deploy:
# Only run if comment is on a PR with the main repo, and if it contains the magic keywords
if: >
contains(github.event.comment.html_url, '/pull/') &&
startsWith(github.event.comment.body, '@delphis-bot create backbone') &&
github.repository == 'poseidon-framework/poseidon-eager'
runs-on: ubuntu-latest
steps:

# indication that the process has started
- name: React on comment
uses: peter-evans/create-or-update-comment@v2
with:
comment-id: ${{ github.event.comment.id }}
reactions: eyes
token: ${{ secrets.delphis_bot_auth_token }}

# Use the @delphis-bot token to check out so we can push later
- uses: actions/checkout@v3
with:
token: ${{ secrets.delphis_bot_auth_token }}

# Action runs on the issue comment, so we don't get the PR by default
# Use the gh cli to check out the PR
- name: Checkout Pull Request
run: gh pr checkout ${{ github.event.issue.number }}
env:
GITHUB_TOKEN: ${{ secrets.delphis_bot_auth_token }}

- uses: actions/setup-node@v3

- name: Get package names from added ssf files
id: get_package_names
run: |
offending_ssfs=''
package_names=''
while read r; do
package_name="$(basename -s .ssf ${r})"
package_dir="$(basename $(dirname ${r}))"
if [[ ${package_name} != ${package_dir} ]]; then
offending_ssfs+="${r} "
fi
package_names+="${package_name} "
done < <( gh api /repos/poseidon-framework/poseidon-eager/pulls/${{ github.event.issue.number }}/files -q '.[].filename' | grep '.ssf$' )
echo "offending_ssfs=${offending_ssfs% }" >>$GITHUB_OUTPUT
echo "package_names=${package_names% }" >>$GITHUB_OUTPUT
env:
GITHUB_TOKEN: ${{ secrets.delphis_bot_auth_token }}

- name: Post comment listing offending packages
if: steps.get_package_names.outputs.offending_ssfs != ''
uses: peter-evans/create-or-update-comment@v2
with:
issue-number: ${{ github.event.issue.number }}
token: ${{ secrets.delphis_bot_auth_token }}
body: |
SSF files MUST be named after their respective package. The following SSF files break this rule:
${{ steps.get_package_names.outputs.offending_ssfs }}
- name: Create eager TSV from SSF
id: create_tsv
if: steps.get_package_names.outputs.offending_ssfs == ''
run: |
for package_name in ${{ steps.get_package_names.outputs.package_names }}; do
./scripts/create_eager_input.sh ${package_name}
done
- name: Add nextflow config from template
id: add_config
if: steps.get_package_names.outputs.offending_ssfs == ''
run: |
for package_name in ${{ steps.get_package_names.outputs.package_names }}; do
cp ./assets/template.config ./packages/${package_name}/${package_name}.config
done
- name: Add tsv_patch from template
id: add_tsv_patch
if: steps.get_package_names.outputs.offending_ssfs == ''
run: |
for package_name in ${{ steps.get_package_names.outputs.package_names }}; do
cp ./assets/template.tsv_patch.sh ./packages/${package_name}/${package_name}.tsv_patch.sh
chmod +x ./packages/${package_name}/${package_name}.tsv_patch.sh
done
- name: Commit & push changes
id: commit_changes
if: steps.create_tsv.outcome == 'success'
run: |
git config user.email "[email protected]"
git config user.name "delphis-bot"
git config push.default upstream
git add .
git status
git commit -m "[automated] Create processing backbone from SSF files"
git push
# # indication that the process is done
# - name: React on comment
# if: steps.commit_changes.outcome == 'success' || steps.commit_changes.outcome == 'failure'
# uses: peter-evans/create-or-update-comment@v2
# with:
# comment-id: ${{ github.event.comment.id }}
# reactions: eyes
7 changes: 7 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"printWidth": 80,
"tabWidth": 2,
"useTabs": false,
"max-line-length": [true, { "limit": 120 }],
"proseWrap": "always"
}
Loading

0 comments on commit c48445a

Please sign in to comment.