Skip to content

Commit

Permalink
Clarified submission instructions for USENIX Security '25 + fixed typ…
Browse files Browse the repository at this point in the history
…os (#85)

Updated and clarified Submission instructions
  • Loading branch information
Aurore54F authored Sep 13, 2024
1 parent 0611ce1 commit e3a7db1
Show file tree
Hide file tree
Showing 4 changed files with 128 additions and 133 deletions.
42 changes: 21 additions & 21 deletions _conferences/usenixsec2025/badges.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
---
title: Badges
order: 30
---

Submitted artifacts could select to be evaluated against the following badges:

<style>
table th:first-of-type {
width: 20%;
}
table th:nth-of-type(2) {
width: 70%;
}
</style>

| USENIX Badges | Description |
|:-------------:|:------------|
| ![available badge]({{ site.baseurl }}/images/usenixbadges-available.svg) | **Artifacts Available:** As per the new Open Science policy of USENIX Security, all accepted papers are mandated to qualify for this badge. To earn this badge, the AEC must judge that the artifacts as described in the Open Science section of the paper have been made available for retrieval, permanently and publicly. The archived copy of the artifacts must be accessible via a long-term stable reference or DOI. For this purpose, we recommend [Zenodo](https://zenodo.org/), but other valid hosting options include institutional and third-party digital repositories (e.g., [FigShare](https://figshare.com/), [Dryad](https://datadryad.org/stash/), [Software Heritage](https://archive.softwareheritage.org/)). Unlike in previous iterations, software development repositories such as [GitHub](https://github.com/), or [GitLab](https://about.gitlab.com/) and personal webpages are **not acceptable** for this badge. Other than making the artifacts available, this badge does not mandate any further requirements on functionality, correctness, or documentation.
| ![functional badge]({{ site.baseurl }}/images/usenixbadges-functional.svg) | **Artifacts Functional:** To earn this badge, the AEC must judge that the artifacts conform to the expectations set by the paper in terms of functionality, usability, and relevance. In short, do the artifacts work and are they useful for producing outcomes associated with the paper? The AEC will consider three aspects of the artifacts in particular. <br>**Documentation**: are the artifacts sufficiently documented to enable them to be exercised by readers of the paper? <br>**Completeness**: do the submitted artifacts include all of the key components described in the paper? <br>**Exercisability**: do the submitted artifacts include the scripts and data needed to run the experiments described in the paper, and can the software be successfully executed? |
| ![reproduced badge]({{ site.baseurl }}/images/usenixbadges-reproduced.svg) | **Results Reproduced:** To earn this badge, the AEC must judge that they can use the submitted artifacts to obtain the main results presented in the paper. In short, is it possible for the AEC to independently repeat the experiments and obtain results that support the main claims made by the paper? The goal of this effort is not to reproduce the results exactly, but instead to generate results independently within an allowed tolerance such that the main claims of the paper are validated. |
---
title: Badges
order: 30
---

Submitted artifacts can be evaluated against the following badges:

<style>
table th:first-of-type {
width: 20%;
}
table th:nth-of-type(2) {
width: 70%;
}
</style>

| USENIX Badges | Description |
|:-------------:|:------------|
| ![available badge]({{ site.baseurl }}/images/usenixbadges-available.svg) | **Artifacts Available:** As per the new "Open-Science policy" of USENIX Security, **all accepted papers are mandated to qualify for this badge**. To earn this badge, the AEC must judge that the artifacts described in the "Open-Science section" of the paper have been **made available for retrieval, permanently and publicly**. The archived copy of the artifacts must be accessible via a long-term stable reference or DOI. For this purpose, we recommend [Zenodo](https://zenodo.org/), but other valid hosting options include institutional and third-party digital repositories (e.g., [FigShare](https://figshare.com/), [Dryad](https://datadryad.org/stash/), or [Software Heritage](https://archive.softwareheritage.org/)). Unlike previous iterations, software development repositories such as [GitHub](https://github.com/), [GitLab](https://about.gitlab.com/), or personal web pages are **not acceptable** for this badge. Other than making the artifacts available, this badge does not mandate any further requirements on functionality, correctness, or documentation.
| ![functional badge]({{ site.baseurl }}/images/usenixbadges-functional.svg) | **Artifacts Functional:** To earn this badge, the AEC must judge that the artifacts **conform to the expectations set by the paper in terms of functionality, usability, and relevance**. In short, do the artifacts work, and are they useful for producing outcomes associated with the paper? The AEC will consider three aspects of the artifacts in particular: <br>**Documentation**: are the artifacts sufficiently documented to enable them to be exercised by readers of the paper? <br>**Completeness**: do the submitted artifacts include all of the key components described in the paper? <br>**Exercisability**: do the submitted artifacts include the scripts and data needed to run the experiments described in the paper, and can the software be successfully executed? |
| ![reproduced badge]({{ site.baseurl }}/images/usenixbadges-reproduced.svg) | **Results Reproduced:** To earn this badge, the AEC must judge that they **can use the submitted artifacts to obtain the main results presented in the paper**. In short, could the AEC independently repeat the experiments and obtain results that support the main claims made by the paper? The goal of this effort is not to reproduce the results exactly but instead to generate results independently within an allowed tolerance such that the main claims of the paper are validated.
3 changes: 2 additions & 1 deletion _conferences/usenixsec2025/change-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@ order: 70

All major changes to the site are documented here for easy tracking:

1. August 19, 2025: First version of the site is created.
1. August 19, 2024: First version of the site is created.
2. September 13, 2024: Some clarifications and minor textual changes.
6 changes: 4 additions & 2 deletions _conferences/usenixsec2025/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ title: Artifact Evaluation
order: 0
---

In pursuit of promoting open science and increasing reproducibility of scientific studies in the security community, the [Call for Papers](https://www.usenix.org/conference/usenixsecurity25/call-for-papers) for USENIX Security '25 initiated an "Open Science" policy. This policy requires authors to share their research artifacts by default which will then be evaluated by an Artifact Evaluation Committee for availability. Optionally, these artifacts can also be assessed for functionality and reproducibility. This site is set up to provide relevant information to authors and artifact evaluators about the artifact evaluation process. At the conclusion of artifact evaluation, this web site will also document the results of the evaluation.
In pursuit of promoting open science and increasing the reproducibility of scientific studies in the security community, the [Call for Papers](https://www.usenix.org/conference/usenixsecurity25/call-for-papers) for USENIX Security '25 initiated an "Open-Science" policy. This policy **requires authors to share their research artifacts by default**, which an Artifact Evaluation Committee will then evaluate for _availability_. Optionally, these artifacts can also be assessed for _functionality and reproducibility_.

All major updates to the site content will be noted in the [Change Log](/usenixsec2025/change-log).
This site is set up to provide relevant information to authors and artifact evaluators about the artifact evaluation process. After the artifact evaluation, this website will document the evaluation results.

All major updates to the site content will be noted in the [Change Log](/usenixsec2025/change-log).
Loading

0 comments on commit e3a7db1

Please sign in to comment.