From c3b7d0d54e58798f782150ea3adc303e3eda04e9 Mon Sep 17 00:00:00 2001 From: cpauvert Date: Tue, 4 Jul 2023 14:00:46 +0200 Subject: [PATCH 01/12] remove legacy jekyll template inclusion fixes #340 --- index.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/index.md b/index.md index b8c9d553..9c0f48c3 100644 --- a/index.md +++ b/index.md @@ -2,8 +2,6 @@ site: sandpaper::sandpaper_site --- -{% include gh\_variables.html %} - Command line interface (CLI) and graphic user interface (GUI) are different ways of interacting with a computer's operating system. They have different pros and cons. Most people are familiar with the GUI as it is the default interface for most software, particularly on Windows and Mac OS. When using the GUI, you see and interact with visual representations of files, folders, applications, and most other functions of your computer. When using the CLI, you work largely with text representations of software, files, folders, input and output. The *shell* is a program that allows you to control your computer by typing instructions on the CLI with a keyboard. From aafd9d0d4548b26e2f1ecb77fade44ee745d26f2 Mon Sep 17 00:00:00 2001 From: Charlie Pauvert Date: Mon, 14 Aug 2023 10:59:23 +0200 Subject: [PATCH 02/12] Fix #339 to correct contributing links (#343) * fix uninformative link spotted by `sandpaper::validate_lesson()` * add correct links for repo and "how to contribute lesson" fixes #339 * Revert "fix uninformative link spotted by `sandpaper::validate_lesson()`" This reverts commit a3d0daec97705dbd52c39b89b2e101f6d1dd6ad2. --- CONTRIBUTING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ec44704c..176edf9d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -101,7 +101,7 @@ community listed at including via social media, slack, newsletters, and email lists. You can also [reach us by email][contact]. -[repo]: https://example.com/FIXME +[repo]: https://github.com/datacarpentry/shell-genomics [contact]: mailto:team@carpentries.org [cp-site]: https://carpentries.org/ [dc-issues]: https://github.com/issues?q=user%3Adatacarpentry @@ -111,7 +111,7 @@ email][contact]. [github]: https://github.com [github-flow]: https://guides.github.com/introduction/flow/ [github-join]: https://github.com/join -[how-contribute]: https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github +[how-contribute]: https://egghead.io/courses/how-to-contribute-to-an-open-source-project-on-github [issues]: https://carpentries.org/help-wanted-issues/ [lc-issues]: https://github.com/issues?q=user%3ALibraryCarpentry [swc-issues]: https://github.com/issues?q=user%3Aswcarpentry From be769bd3ceb250ebea57c2270b31fa9b640743e7 Mon Sep 17 00:00:00 2001 From: Charlie Pauvert Date: Tue, 15 Aug 2023 10:08:10 +0200 Subject: [PATCH 03/12] fix uninformative link spotted by `sandpaper::validate_lesson()` (#342) * fix uninformative link spotted by `sandpaper::validate_lesson()` * use the full name of the lesson in link to the notes of logging into remote servers --------- Co-authored-by: Paul Smith --- episodes/01-introduction.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/episodes/01-introduction.md b/episodes/01-introduction.md index 18aeca11..fe5b6ccb 100644 --- a/episodes/01-introduction.md +++ b/episodes/01-introduction.md @@ -65,8 +65,7 @@ would take up time even more time. A 'ready-to-go' server lets us focus on learn ## How to access the remote server -You can log-in to the remote server using the instructions -[here](https://datacarpentry.org/cloud-genomics/02-logging-onto-cloud#logging-onto-a-cloud-instance). +You can log-in to the remote server using the [instructions from the Introduction to Cloud Computing for Genomics lesson](https://datacarpentry.org/cloud-genomics/02-logging-onto-cloud#logging-onto-a-cloud-instance). Your instructor will supply to you the `ip_address` and password that you need to login. Each of you will have a different `ip_address`. This will From e19569a8ed2a09a6c54343473d1138cb45ffefe5 Mon Sep 17 00:00:00 2001 From: Robert Davey Date: Tue, 29 Aug 2023 14:30:41 +0100 Subject: [PATCH 04/12] Update slack links (#349) --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c4518a3f..61da97b4 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.3260560.svg)](https://doi.org/10.5281/zenodo.3260560) -[![Create a Slack Account with us](https://img.shields.io/badge/Create_Slack_Account-The_Carpentries-071159.svg)](https://swc-slack-invite.herokuapp.com/) -[![Slack Status](https://img.shields.io/badge/Slack_Channel-dc--genomics-E01563.svg)](https://swcarpentry.slack.com/messages/C9N1K7DCY) +[![Create a Slack Account with us](https://img.shields.io/badge/Create_Slack_Account-The_Carpentries-071159.svg)](https://slack-invite.carpentries.org/) +[![Slack Status](https://img.shields.io/badge/Slack_Channel-dc--genomics-E01563.svg)](https://carpentries.slack.com/messages/C9N1K7DCY) # Shell Genomics lessons From d95406e3d6f6640208b1da3fb987a6e4ad36715e Mon Sep 17 00:00:00 2001 From: Graeme Grimes Date: Fri, 8 Sep 2023 10:17:01 +0100 Subject: [PATCH 05/12] Create etherpad_template.md added etherpad template for shell genomics lesson --- instructors/etherpad_template.md | 200 +++++++++++++++++++++++++++++++ 1 file changed, 200 insertions(+) create mode 100644 instructors/etherpad_template.md diff --git a/instructors/etherpad_template.md b/instructors/etherpad_template.md new file mode 100644 index 00000000..ad46dd46 --- /dev/null +++ b/instructors/etherpad_template.md @@ -0,0 +1,200 @@ +--- +title: Etherpad template +subtitle: Template text to paste in to collaborative document +--- + +# Introduction to the Command Line for Genomic + +**Date** + +**Instructor** + +**Helper** + +**Website** +[https://datacarpentry.org/shell-genomics/instructor/index.html](https://datacarpentry.org/shell-genomics/instructor/index.html) + +## Timings - EDIT FOR YOUR WORKSHOP + + +- Introducing the Shell 09:30-10:00 (30 mins) +- Navigating Files and Directories 10:00-10:50 (50 mins) +- Break 10:50-11:05 (15 mins) +- Working with Files and Directories 11:05-11:50 (45 mins) +- Redirection 11:50-12:35 (45 mins) +- Break 12:35-13:30 (55 mins) +- Writing Scripts and Working with Data 13:30-14:10 (40 mins) +- Project Organization 14:10-14:40 (30 mins) +- Wrap up and feedback 14:40-14:55 (15 mins) + +## Attendees + +Please write your name below to confirm attendance, and one thing you're hoping to get out of this workshop: + +1. 2. 3. 4. 5. +## Feedback + +Link to feedback form if needed + +## [Introducing the Shell](https://datacarpentry.org/shell-genomics/01-introduction.html) + +### Exercise: + +Use the `-l` option for the `ls` command to display more information for each item in the directory. + +What is one piece of additional information this long format gives you that you don’t see with the bare `ls` command? +- - - + +## [Navigating Files and Directories](https://datacarpentry.org/shell-genomics/02-the-filesystem.html) + +### Exercise: FINDING HIDDEN DIRECTORIES + +First navigate to the `shell_data` directory. +There is a hidden directory within this directory. Explore the options for `ls` to find out how to see hidden directories. +List the contents of the directory and identify the name of the text file in that directory. + +**Hint:** hidden files and folders in Unix start with ., for example .my_hidden_directory + + +### Exercise: NAVIGATING PRACTICE + +Navigate to your home directory. From there, list the contents of the `untrimmed_fastq` directory. + +### Exercise: RELATIVE PATH RESOLUTION + +Using the filesystem diagram on the lesson page [below](https://datacarpentry.org/shell-genomics/02-the-filesystem.html), if pwd `displays` `/Users/thing`, what will ls .`./backup display`? + +Put a + next to the answer + +1. ../backup: No such file or directory +2. 2012-12-01 2013-01-08 2013-01-27 +3. 2012-12-01/ 2013-01-08/ 2013-01-27/ +4. original pnas_final pnas_sub + + +## [Working with Files and Directories](https://datacarpentry.org/shell-genomics/03-working-with-files.html) + +### Exercise: + +Do each of the following tasks from your current directory using a single `ls` command for each: + +1. List all of the files in /usr/bin that start with the letter ‘c’. +2. List all of the files in /usr/bin that contain the letter ‘a’. +3. List all of the files in /usr/bin that end with the letter ‘o’. + +Bonus: List all of the files in /usr/bin that contain the letter ‘a’ or the letter ‘c’. + +**Hint:** The bonus question requires a Unix wildcard that we haven’t talked about yet. Try searching the internet for information about Unix wildcards to find what you need to solve the bonus problem. +- - - +### Exercise: echo and wildcards + +`echo` is a built-in shell command that writes its arguments, like a line of text to standard output. +The `echo` command can also be used with pattern matching characters, such as wildcard characters. +Here we will use the `echo` command to see how the wildcard character is interpreted by the shell. + +```bash +$ echo *.fastq +``` +What would the output look like if the wildcard could not be matched? + +Compare the outputs of + +1. `echo *.missing` +2. `ls *.missing` +- - - +### Exercise: command history + +Find the line number in your `history` for the command that listed all the `.sh` files in `/usr/bin`. +Rerun that command. + +- - - + +### Exercise: Examining Files + +1. Print out the contents of the `~/shell_data/untrimmed_fastq/SRR097977.fastq` file. What is the last line of the file? + +2. From your home directory, and without changing directories, use one short command to print the contents of all of the files in the +`~/shell_data/untrimmed_fastq` directory. + +- - - + +### Exercise: Examining Files + +Use `less` on the file SRR097977.fastq and find the next three nucleotides (characters) after the first instance of the sequence `TTTTT`? + +- - - + +### Exercise: + +Starting in the `shell_data/untrimmed_fastq/` directory, do the following: + +1. Make sure that you have deleted your backup directory and all files it contains. +2. Create a backup of each of your FASTQ files using cp. (Note: You’ll need to do this individually for each of the two FASTQ files. We haven’t learned yet how to do this with a wildcard.) +3. Use a wildcard to move all of your backup files to a new backup directory. +4. Change the permissions on all of your backup files to be write-protected. + +- - - + +## [Redirection](https://datacarpentry.org/shell-genomics/04-redirection.html) + +### EXERCISE: + +1. Search for the sequence `GNATNACCACTTCC in the `SRR098026.fastq` file. Have your search return all matching lines and the name (or identifier) for each sequence that contains a match. + +2. Search for the sequence `AAGTT` in both FASTQ files. Have your search return all matching lines and the name (or identifier) for each sequence that contains a match. +- - - + +### EXERCISE + +How many sequences are there in `SRR098026.fastq`? Remember that every sequence is formed by four lines. + +- - - + +### EXERCISE + +How many sequences in `SRR098026.fastq` contain at least 3 consecutive Ns? + +- - - + +### EXERCISE + +Print the file prefix of all of the `.txt` files in our current directory. + +- - - + +### EXERCISE + +Remove `_2019` from all of the `.txt` files. + +- - - + +## [Writing Scripts and Working with Data](https://datacarpentry.org/shell-genomics/05-writing-scripts.html) + +### EXERCISE + +Open `README.txt` and add the date to the top of the file and save the file. + +- - - + +### EXERCISE + +We want the script to tell us when it’s done. + +Open `bad-reads-script.sh` and add the line echo `"Script finished!"` after the grep command and save the file. +Run the updated script. + +- - - + +## [Project Organization[(https://datacarpentry.org/shell-genomics/06-organization.html) + +## Evaluation and Feedback + +Instructors and lesson maintainers use these responses to improve the lesson. +It's very helpful! + +#### Please list one thing you liked or found particularly useful + +- - - - - +#### Please list another thing you found less useful, or that could be improved + +- - - - - From ebafe77ebf3b30f7200f0fa244e01961d95ee053 Mon Sep 17 00:00:00 2001 From: Graeme Grimes Date: Fri, 8 Sep 2023 10:18:57 +0100 Subject: [PATCH 06/12] Update etherpad_template.md added exercises for Project Organization --- instructors/etherpad_template.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/instructors/etherpad_template.md b/instructors/etherpad_template.md index ad46dd46..44992a5b 100644 --- a/instructors/etherpad_template.md +++ b/instructors/etherpad_template.md @@ -187,6 +187,21 @@ Run the updated script. ## [Project Organization[(https://datacarpentry.org/shell-genomics/06-organization.html) +### EXERCISE +Use the `mkdir` command to make the following directories: + +``` +dc_workshop +dc_workshop/docs +dc_workshop/data +dc_workshop/results +``` +- - - + +### EXERCISE + +Using your knowledge of the shell, use the append redirect >> to create a file called `dc_workshop_log_XXXX_XX_XX.sh` (Use the four-digit year, two-digit month, and two digit day, e.g. `dc_workshop_log_2017_10_27.sh`) + ## Evaluation and Feedback Instructors and lesson maintainers use these responses to improve the lesson. From 5c80eeb2725d8f082415f022935f570d630d94ed Mon Sep 17 00:00:00 2001 From: Graeme Grimes Date: Fri, 8 Sep 2023 10:28:38 +0100 Subject: [PATCH 07/12] Update etherpad_template.md updated some formatting --- instructors/etherpad_template.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/instructors/etherpad_template.md b/instructors/etherpad_template.md index 44992a5b..2f51eb69 100644 --- a/instructors/etherpad_template.md +++ b/instructors/etherpad_template.md @@ -64,7 +64,7 @@ Navigate to your home directory. From there, list the contents of the `untrimmed Using the filesystem diagram on the lesson page [below](https://datacarpentry.org/shell-genomics/02-the-filesystem.html), if pwd `displays` `/Users/thing`, what will ls .`./backup display`? -Put a + next to the answer +Put a `+` next to the answer you think is correct. 1. ../backup: No such file or directory 2. 2012-12-01 2013-01-08 2013-01-27 @@ -78,11 +78,11 @@ Put a + next to the answer Do each of the following tasks from your current directory using a single `ls` command for each: -1. List all of the files in /usr/bin that start with the letter ‘c’. -2. List all of the files in /usr/bin that contain the letter ‘a’. -3. List all of the files in /usr/bin that end with the letter ‘o’. +1. List all of the files in `/usr/bin` that start with the letter ‘c’. +2. List all of the files in `/usr/bin` that contain the letter ‘a’. +3. List all of the files in `/usr/bin` that end with the letter ‘o’. -Bonus: List all of the files in /usr/bin that contain the letter ‘a’ or the letter ‘c’. +Bonus: List all of the files in `/usr/bin` that contain the letter ‘a’ or the letter ‘c’. **Hint:** The bonus question requires a Unix wildcard that we haven’t talked about yet. Try searching the internet for information about Unix wildcards to find what you need to solve the bonus problem. - - - From 71ec637b24ec9868888b9866e02940b6e7871b98 Mon Sep 17 00:00:00 2001 From: Graeme Grimes Date: Fri, 8 Sep 2023 10:29:14 +0100 Subject: [PATCH 08/12] Update etherpad_template.md --- instructors/etherpad_template.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/instructors/etherpad_template.md b/instructors/etherpad_template.md index 2f51eb69..64f1e43d 100644 --- a/instructors/etherpad_template.md +++ b/instructors/etherpad_template.md @@ -202,6 +202,8 @@ dc_workshop/results Using your knowledge of the shell, use the append redirect >> to create a file called `dc_workshop_log_XXXX_XX_XX.sh` (Use the four-digit year, two-digit month, and two digit day, e.g. `dc_workshop_log_2017_10_27.sh`) +- - - + ## Evaluation and Feedback Instructors and lesson maintainers use these responses to improve the lesson. @@ -210,6 +212,7 @@ It's very helpful! #### Please list one thing you liked or found particularly useful - - - - - + #### Please list another thing you found less useful, or that could be improved - - - - - From 1b7b4b6cabed943957f2226a66255e02d8c1a2c1 Mon Sep 17 00:00:00 2001 From: Graeme Grimes Date: Fri, 8 Sep 2023 10:50:53 +0100 Subject: [PATCH 09/12] Update etherpad_template.md --- instructors/etherpad_template.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/instructors/etherpad_template.md b/instructors/etherpad_template.md index 64f1e43d..e8b7bbe4 100644 --- a/instructors/etherpad_template.md +++ b/instructors/etherpad_template.md @@ -16,6 +16,7 @@ subtitle: Template text to paste in to collaborative document ## Timings - EDIT FOR YOUR WORKSHOP +Here's a tentative schedule for the workshop - Introducing the Shell 09:30-10:00 (30 mins) - Navigating Files and Directories 10:00-10:50 (50 mins) @@ -29,7 +30,7 @@ subtitle: Template text to paste in to collaborative document ## Attendees -Please write your name below to confirm attendance, and one thing you're hoping to get out of this workshop: +Please write your name below to confirm your attendance and mention one thing you hope to learn from this workshop. 1. 2. 3. 4. 5. ## Feedback @@ -172,7 +173,7 @@ Remove `_2019` from all of the `.txt` files. ### EXERCISE -Open `README.txt` and add the date to the top of the file and save the file. +Open README.txt, add the date to the top of the file, and then save it. - - - @@ -185,7 +186,7 @@ Run the updated script. - - - -## [Project Organization[(https://datacarpentry.org/shell-genomics/06-organization.html) +## [Project Organization](https://datacarpentry.org/shell-genomics/06-organization.html) ### EXERCISE Use the `mkdir` command to make the following directories: @@ -206,8 +207,7 @@ Using your knowledge of the shell, use the append redirect >> to create a file c ## Evaluation and Feedback -Instructors and lesson maintainers use these responses to improve the lesson. -It's very helpful! +Your feedback is valuable as it helps instructors and lesson maintainers enhance the content. #### Please list one thing you liked or found particularly useful From 3f678262a01fbd3a826dc58bcd26f278b3c61730 Mon Sep 17 00:00:00 2001 From: Graeme Grimes Date: Fri, 8 Sep 2023 10:51:36 +0100 Subject: [PATCH 10/12] Update etherpad_template.md --- instructors/etherpad_template.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/instructors/etherpad_template.md b/instructors/etherpad_template.md index e8b7bbe4..1358e0e6 100644 --- a/instructors/etherpad_template.md +++ b/instructors/etherpad_template.md @@ -32,7 +32,11 @@ Here's a tentative schedule for the workshop Please write your name below to confirm your attendance and mention one thing you hope to learn from this workshop. -1. 2. 3. 4. 5. +1. +2. +3. +4. + ## Feedback Link to feedback form if needed From 84a365a0f7158e50f1663f40c807b62a1726f172 Mon Sep 17 00:00:00 2001 From: Graeme Grimes Date: Fri, 8 Sep 2023 11:00:32 +0100 Subject: [PATCH 11/12] Update etherpad_template.md changed link to website materials from instructors view --- instructors/etherpad_template.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/instructors/etherpad_template.md b/instructors/etherpad_template.md index 1358e0e6..8dca1c47 100644 --- a/instructors/etherpad_template.md +++ b/instructors/etherpad_template.md @@ -12,7 +12,7 @@ subtitle: Template text to paste in to collaborative document **Helper** **Website** -[https://datacarpentry.org/shell-genomics/instructor/index.html](https://datacarpentry.org/shell-genomics/instructor/index.html) +[https://datacarpentry.org/shell-genomics](https://datacarpentry.org/shell-genomics) ## Timings - EDIT FOR YOUR WORKSHOP From 09f551ef9bb49c2facbabc7fa85c25edb0219197 Mon Sep 17 00:00:00 2001 From: "The Carpentries Apprentice (beta)" <64428345+carpentries-bot@users.noreply.github.com> Date: Wed, 13 Sep 2023 14:53:15 -0400 Subject: [PATCH 12/12] [actions] update sandpaper workflow to version 0.13.0 (#351) Update Workflows from sandpaper version 0.11.16 -> 0.13.0 Co-authored-by: zkamvar --- .github/workflows/README.md | 2 +- .github/workflows/sandpaper-version.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 101967e4..d6edf88d 100755 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -96,7 +96,7 @@ are okay. This update is run ~~weekly or~~ on demand. -### 03 Maintain: Update Pacakge Cache (update-cache.yaml) +### 03 Maintain: Update Package Cache (update-cache.yaml) For lessons that have generated content, we use {renv} to ensure that the output is stable. This is controlled by a single lockfile which documents the packages diff --git a/.github/workflows/sandpaper-version.txt b/.github/workflows/sandpaper-version.txt index db07b54a..54d1a4f2 100644 --- a/.github/workflows/sandpaper-version.txt +++ b/.github/workflows/sandpaper-version.txt @@ -1 +1 @@ -0.11.16 +0.13.0