From f9898805ccf32dcdcf4d0441385bde50cf102be1 Mon Sep 17 00:00:00 2001 From: Amir Mehrkar Date: Thu, 30 Sep 2021 14:17:48 +0100 Subject: [PATCH 1/6] Update releasing-files.md Addition of information to clarify with who data released from Level 4 can be shared and to reinforce researchers must obtain publication (aka exit approval) for studies from NHS England and how to obtain this. --- docs/releasing-files.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/releasing-files.md b/docs/releasing-files.md index eeaca7202..4a7e3b108 100644 --- a/docs/releasing-files.md +++ b/docs/releasing-files.md @@ -7,11 +7,19 @@ You will find your files to review in `D:\Level4Files\workspaces\ Level 4 outputs **can only be released by specific members of the OpenSAFELY team**. When you are ready to request a release of your aggregated results, and only the minimum outputs necessary (for which you have also applied any necessary disclosure controls, such as small number suppression) **please [complete this form](documents/OpenSAFELY_Output_Review_Form.docx) and email us at **. +Once these Level 4 outputs have been released, you can share the data **in confidence** with wider members of the research team, including external collaborators. For example, the study author may copy the outputs from the GitHub repository to a document being prepared for publication, or to seek feedback and contribution from other colleagues to inform the final paper or report. + +However, prior to any open publication (e.g. MedRxiv), or circulation of the findings outside of the immediate research team or collaborators (e.g. presentations given to interested groups), **the final paper/report/presentation (etc.), including a brief lay summary of the important conclusions, MUST FIRST BE EMAILED to amir.mehrkar@phc.ox.ac.uk who will seek final approval for release from NHS England.** The purpose of this approval step is to check that the paper/report aligns with the stated application purpose. + + !!! warning ** Data may only be shared with individuals without Level 4 access after they have been released outside of the secure area. You MUST NOT screen share (or share via any other means) any results that have not been released through the official process.** !!! warning ** DO NOT release your aggregated study results to your GitHub repository yourself (unless you have been given permission by OpenSAFELY).** + +!!! warning + ** YOU MUST NOT publish any results (or related content) with journals or online, or circulate findings (e.g. presentations) with any interested groups outside of the immediate research team, **until you have received approval from NHS England.** To obtain approval email with your paper/report/presentation including a brief lay summary of the important conclusions. !!! note Individual researchers who have Level 4 access have responsibility for redacting sensitive information or choosing not to publish it at all. The study author should do everything they can to make this easy; for example, carrying out low number suppression automatically, documenting code clearly, and only selecting essential items for publication when deciding what to label as `moderately_sensitive`. From 2b81d4a64ae33b3a4f8ffd4242f632cc3776a71d Mon Sep 17 00:00:00 2001 From: Amir Mehrkar Date: Thu, 30 Sep 2021 14:24:21 +0100 Subject: [PATCH 2/6] Update releasing-files.md --- docs/releasing-files.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/releasing-files.md b/docs/releasing-files.md index 4a7e3b108..6ab60f3c8 100644 --- a/docs/releasing-files.md +++ b/docs/releasing-files.md @@ -13,7 +13,7 @@ However, prior to any open publication (e.g. MedRxiv), or circulation of the fin !!! warning - ** Data may only be shared with individuals without Level 4 access after they have been released outside of the secure area. You MUST NOT screen share (or share via any other means) any results that have not been released through the official process.** + ** Data may only be shared **in confidence** with wider members of the research team, including external collaborators, **after** they have been released outside of the secure Level 4 area. You MUST NOT screen share (or share via any other means) any results that have not been released through the official process.** !!! warning ** DO NOT release your aggregated study results to your GitHub repository yourself (unless you have been given permission by OpenSAFELY).** From 29e3ab2042660009780cff0a21087086df51c5ef Mon Sep 17 00:00:00 2001 From: Amir Mehrkar Date: Thu, 30 Sep 2021 16:50:24 +0100 Subject: [PATCH 3/6] Update releasing-files.md --- docs/releasing-files.md | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/docs/releasing-files.md b/docs/releasing-files.md index 6ab60f3c8..1264c9fac 100644 --- a/docs/releasing-files.md +++ b/docs/releasing-files.md @@ -5,21 +5,13 @@ The final tier is review of so-called "Level 4" outputs, where the OpenSAFELY fr If you have Level 4 access you can log in and review your aggregated results, and perform any necessary redactions before requesting files to be released. You will find your files to review in `D:\Level4Files\workspaces\`. -Level 4 outputs **can only be released by specific members of the OpenSAFELY team**. When you are ready to request a release of your aggregated results, and only the minimum outputs necessary (for which you have also applied any necessary disclosure controls, such as small number suppression) **please [complete this form](documents/OpenSAFELY_Output_Review_Form.docx) and email us at **. - -Once these Level 4 outputs have been released, you can share the data **in confidence** with wider members of the research team, including external collaborators. For example, the study author may copy the outputs from the GitHub repository to a document being prepared for publication, or to seek feedback and contribution from other colleagues to inform the final paper or report. - -However, prior to any open publication (e.g. MedRxiv), or circulation of the findings outside of the immediate research team or collaborators (e.g. presentations given to interested groups), **the final paper/report/presentation (etc.), including a brief lay summary of the important conclusions, MUST FIRST BE EMAILED to amir.mehrkar@phc.ox.ac.uk who will seek final approval for release from NHS England.** The purpose of this approval step is to check that the paper/report aligns with the stated application purpose. - +Level 4 outputs **can only be released by specific members of the OpenSAFELY team**. When you are ready to request an OUTPUT REVIEW and RELEASE of your aggregated results, and only the minimum outputs necessary (for which you have also applied any necessary disclosure controls, such as small number suppression) **please [complete this form](documents/OpenSAFELY_Output_Review_Form.docx) and email us at **. !!! warning - ** Data may only be shared **in confidence** with wider members of the research team, including external collaborators, **after** they have been released outside of the secure Level 4 area. You MUST NOT screen share (or share via any other means) any results that have not been released through the official process.** + ** Data may only be shared **in confidence** with wider members of the research team, including external collaborators, **after** they have been released outside of the secure Level 4 area. You MUST NOT screen share (or share via any other means) any results from the Level 4 environment that have not been submitted for OUTPUT REVIEW.** !!! warning ** DO NOT release your aggregated study results to your GitHub repository yourself (unless you have been given permission by OpenSAFELY).** - -!!! warning - ** YOU MUST NOT publish any results (or related content) with journals or online, or circulate findings (e.g. presentations) with any interested groups outside of the immediate research team, **until you have received approval from NHS England.** To obtain approval email with your paper/report/presentation including a brief lay summary of the important conclusions. !!! note Individual researchers who have Level 4 access have responsibility for redacting sensitive information or choosing not to publish it at all. The study author should do everything they can to make this easy; for example, carrying out low number suppression automatically, documenting code clearly, and only selecting essential items for publication when deciding what to label as `moderately_sensitive`. @@ -49,6 +41,20 @@ It may be possible for exceptions to be made if they can be justified as being b **If you are unsure about anything, please email us: .** +## Sharing and Publication of your study results (released the Level 4 environment) and GitHub repository + +### Study results +Once the Level 4 outputs have been released (as per the above process), you can share the data **in confidence** with wider members of the research team, including external collaborators. For example, the study author may copy the outputs from the GitHub repository to a document being prepared for publication, or to seek feedback and contribution from other colleagues to inform the final paper or report. + +However, prior to any open publication (e.g. MedRxiv), or circulation of the findings outside of the immediate research team or collaborators (e.g. presentations given to interested groups), the final paper/report/presentation (etc.), including a brief lay summary of the important conclusions, **MUST FIRST BE EMAILED to who will seek final approval for release from NHS England.** The purpose of this approval step is to check that the paper/report aligns with the stated application purpose. + +### Study GitHub repository + +Similarly, **the study repository must be kept private until project completion and NHS England approval has been obtained**; however, the OpenSAFELY Oversight Board has recommended that all study GitHub repositories (to include the study definition code, codelists and released results) be make public 12 months after any code has been executed against the underlying patient data. + +!!! warning + ** YOU MUST NOT publish any results or circulate findings with any interested groups outside of the immediate research team, or make public your study GitHub repository **until you have received approval from NHS England.** To obtain approval email with your paper/report/presentation including a brief lay summary of the important conclusions. + ## Redaction data breaches and how to recover from them ### The issues From ed6fed65e412924f5fb53b7f50feb565630c4f81 Mon Sep 17 00:00:00 2001 From: Amir Mehrkar Date: Thu, 30 Sep 2021 17:06:10 +0100 Subject: [PATCH 4/6] Update releasing-files.md --- docs/releasing-files.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/releasing-files.md b/docs/releasing-files.md index 1264c9fac..a39ddcf5e 100644 --- a/docs/releasing-files.md +++ b/docs/releasing-files.md @@ -8,7 +8,7 @@ You will find your files to review in `D:\Level4Files\workspaces\ Level 4 outputs **can only be released by specific members of the OpenSAFELY team**. When you are ready to request an OUTPUT REVIEW and RELEASE of your aggregated results, and only the minimum outputs necessary (for which you have also applied any necessary disclosure controls, such as small number suppression) **please [complete this form](documents/OpenSAFELY_Output_Review_Form.docx) and email us at **. !!! warning - ** Data may only be shared **in confidence** with wider members of the research team, including external collaborators, **after** they have been released outside of the secure Level 4 area. You MUST NOT screen share (or share via any other means) any results from the Level 4 environment that have not been submitted for OUTPUT REVIEW.** + ** Data may only be shared in confidence with wider members of the research team, including external collaborators, after they have been released outside of the secure Level 4 area. You MUST NOT screen share (or share via any other means) any results from the Level 4 environment that have not been submitted for OUTPUT REVIEW.** !!! warning ** DO NOT release your aggregated study results to your GitHub repository yourself (unless you have been given permission by OpenSAFELY).** @@ -41,19 +41,19 @@ It may be possible for exceptions to be made if they can be justified as being b **If you are unsure about anything, please email us: .** -## Sharing and Publication of your study results (released the Level 4 environment) and GitHub repository +## Sharing and Publication of your study results (released from the Level 4 environment) and GitHub repository ### Study results Once the Level 4 outputs have been released (as per the above process), you can share the data **in confidence** with wider members of the research team, including external collaborators. For example, the study author may copy the outputs from the GitHub repository to a document being prepared for publication, or to seek feedback and contribution from other colleagues to inform the final paper or report. -However, prior to any open publication (e.g. MedRxiv), or circulation of the findings outside of the immediate research team or collaborators (e.g. presentations given to interested groups), the final paper/report/presentation (etc.), including a brief lay summary of the important conclusions, **MUST FIRST BE EMAILED to who will seek final approval for release from NHS England.** The purpose of this approval step is to check that the paper/report aligns with the stated application purpose. +However, prior to any open publication (e.g. MedRxiv), or circulation of the findings outside of the immediate research team or collaborators (e.g. presentations given to interested groups), the final paper/report/presentation (etc.), including a brief lay summary of the important conclusions, **must first be emailed to who will seek final approval for release from NHS England.** The purpose of this approval step is to check that the paper/report aligns with the stated application purpose. ### Study GitHub repository Similarly, **the study repository must be kept private until project completion and NHS England approval has been obtained**; however, the OpenSAFELY Oversight Board has recommended that all study GitHub repositories (to include the study definition code, codelists and released results) be make public 12 months after any code has been executed against the underlying patient data. !!! warning - ** YOU MUST NOT publish any results or circulate findings with any interested groups outside of the immediate research team, or make public your study GitHub repository **until you have received approval from NHS England.** To obtain approval email with your paper/report/presentation including a brief lay summary of the important conclusions. + ** YOU MUST NOT publish any results or circulate findings with any interested groups outside of the immediate research team, or make public your study GitHub repository, until you have received approval from NHS England. To obtain approval email with your paper/report/presentation including a brief lay summary of the important conclusions. ## Redaction data breaches and how to recover from them From 811e368a68bd166cdefbb2a1ee5744a7bd37e4a6 Mon Sep 17 00:00:00 2001 From: Amir Mehrkar Date: Thu, 30 Sep 2021 17:19:42 +0100 Subject: [PATCH 5/6] Update releasing-files.md --- docs/releasing-files.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/releasing-files.md b/docs/releasing-files.md index a39ddcf5e..ec25296ff 100644 --- a/docs/releasing-files.md +++ b/docs/releasing-files.md @@ -53,7 +53,7 @@ However, prior to any open publication (e.g. MedRxiv), or circulation of the fin Similarly, **the study repository must be kept private until project completion and NHS England approval has been obtained**; however, the OpenSAFELY Oversight Board has recommended that all study GitHub repositories (to include the study definition code, codelists and released results) be make public 12 months after any code has been executed against the underlying patient data. !!! warning - ** YOU MUST NOT publish any results or circulate findings with any interested groups outside of the immediate research team, or make public your study GitHub repository, until you have received approval from NHS England. To obtain approval email with your paper/report/presentation including a brief lay summary of the important conclusions. + ** YOU MUST NOT publish any results or circulate findings with any interested groups outside of the immediate research team / collaborators, or make public your study GitHub repository, until you have received approval from NHS England. To obtain approval email with your paper/report/presentation including a brief lay summary of the important conclusions.** ## Redaction data breaches and how to recover from them From 76cc00810eff2b2f4d5fe01b44bdd94729e270af Mon Sep 17 00:00:00 2001 From: Amir Mehrkar Date: Thu, 30 Sep 2021 17:27:46 +0100 Subject: [PATCH 6/6] Update releasing-files.md --- docs/releasing-files.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/releasing-files.md b/docs/releasing-files.md index ec25296ff..d248a4110 100644 --- a/docs/releasing-files.md +++ b/docs/releasing-files.md @@ -8,7 +8,7 @@ You will find your files to review in `D:\Level4Files\workspaces\ Level 4 outputs **can only be released by specific members of the OpenSAFELY team**. When you are ready to request an OUTPUT REVIEW and RELEASE of your aggregated results, and only the minimum outputs necessary (for which you have also applied any necessary disclosure controls, such as small number suppression) **please [complete this form](documents/OpenSAFELY_Output_Review_Form.docx) and email us at **. !!! warning - ** Data may only be shared in confidence with wider members of the research team, including external collaborators, after they have been released outside of the secure Level 4 area. You MUST NOT screen share (or share via any other means) any results from the Level 4 environment that have not been submitted for OUTPUT REVIEW.** + ** Data may only be shared IN CONFIDENCE with wider members of the research team, including external collaborators, after they have been released outside of the secure Level 4 area. You MUST NOT screen share (or share via any other means) any results from the Level 4 environment that have not been submitted for OUTPUT REVIEW. See below for publication guidance.** !!! warning ** DO NOT release your aggregated study results to your GitHub repository yourself (unless you have been given permission by OpenSAFELY).** @@ -41,19 +41,19 @@ It may be possible for exceptions to be made if they can be justified as being b **If you are unsure about anything, please email us: .** -## Sharing and Publication of your study results (released from the Level 4 environment) and GitHub repository +## Publication of your study results (released from the Level 4 environment) and GitHub repository ### Study results Once the Level 4 outputs have been released (as per the above process), you can share the data **in confidence** with wider members of the research team, including external collaborators. For example, the study author may copy the outputs from the GitHub repository to a document being prepared for publication, or to seek feedback and contribution from other colleagues to inform the final paper or report. -However, prior to any open publication (e.g. MedRxiv), or circulation of the findings outside of the immediate research team or collaborators (e.g. presentations given to interested groups), the final paper/report/presentation (etc.), including a brief lay summary of the important conclusions, **must first be emailed to who will seek final approval for release from NHS England.** The purpose of this approval step is to check that the paper/report aligns with the stated application purpose. +However, **for publication (e.g. MedRxiv), or circulation of the findings outside of the immediate research team or collaborators** (e.g. presentations given to interested groups), the final paper/report/presentation (etc.), including a brief lay summary of the important conclusions, **must first be emailed to who will seek final approval for release from NHS England.** The purpose of this approval step is to check that the paper/report aligns with the stated application purpose. ### Study GitHub repository Similarly, **the study repository must be kept private until project completion and NHS England approval has been obtained**; however, the OpenSAFELY Oversight Board has recommended that all study GitHub repositories (to include the study definition code, codelists and released results) be make public 12 months after any code has been executed against the underlying patient data. !!! warning - ** YOU MUST NOT publish any results or circulate findings with any interested groups outside of the immediate research team / collaborators, or make public your study GitHub repository, until you have received approval from NHS England. To obtain approval email with your paper/report/presentation including a brief lay summary of the important conclusions.** + ** YOU MUST NOT publish any results or circulate findings with any interested groups outside of the immediate research team / collaborators, or make public your study GitHub repository, until you have received approval from NHS England. To obtain approval email with your paper/report/presentation including a brief lay summary of the important conclusions.** ## Redaction data breaches and how to recover from them