Skip to content

Commit

Permalink
Edits to submission instructions, badges, and organizers (#74)
Browse files Browse the repository at this point in the history
  • Loading branch information
dcdelia authored Jun 15, 2024
1 parent 0e8e4b7 commit fd4ba4c
Show file tree
Hide file tree
Showing 4 changed files with 6 additions and 6 deletions.
2 changes: 1 addition & 1 deletion _conferences/ndss2025/badges.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ order: 10

Authors can request their artifact to be evaluated towards one, two, or all of the following badges:

* **Available.** To earn this badge, the AEC must judge that the artifact associated with the paper has been made available for retrieval *permanently and publicly*. However, an artifact undergoing AE often evolves as a consequence of the AEC feedback it receives. Therefore, if authors opt for "volatile" storage for the initial submission, they must upload the final materials to public services with permanent storage before artifact notification deadline. We encourage authors to consider services such as Zenodo, FigShare, or Dryad that also create a citable DOI, which the final artifact appendix version for publication must mention. For this badge, a README file explicitly referencing the paper and a LICENSE file should be provided.
* **Available.** To earn this badge, the AEC must judge that the artifact associated with the paper has been made available for retrieval *permanently and publicly*. As an artifact undergoing AE often evolves as a consequence of AEC feedback, authors can use mutable storage for the initial submission, but must commit to uploading their materials to public services (e.g., Zenodo, FigShare, Dryad) for permanent storage backed by a Digital Object Identifier (DOI) if the badge is awarded. The artifact appendix prepared for publication will have to mention the artifact DOI. Authors are welcome to report additional sources, like GitHub and GitLab, that may ease the dissemination of the artifact and possible future updates. Furthermore, for this badge, authors should provide a `README` file referencing the paper and a `LICENSE` file for the materials.

* **Functional.** To earn this badge, the AEC must judge that the artifact conforms to the expectations set by the paper for functionality, usability, and relevance. Also, an artifact must be usable on *other machines* than the authors’, including when specialized hardware is required (for example, paths, addresses, and identifiers must not be hardcoded.) The AEC will particularly consider three aspects:
* *Documentation*: is the artifact sufficiently documented to be exercised by readers of the paper?
Expand Down
4 changes: 2 additions & 2 deletions _conferences/ndss2025/call.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,13 @@ Submission page: [https://ndss25ae-summer.hotcrp.com/](https://ndss25ae-summer.h
* Camera-ready deadline for papers: Thu, 5 Dec 2024


Submission page: TBA
Submission page: [https://ndss25ae-fall.hotcrp.com/](https://ndss25ae-fall.hotcrp.com/)

## Evaluation process

Authors are invited to submit artifacts soon after receiving the paper notification. At least one contact author must be reachable and respond to questions in a timely manner during the entire evaluation period to allow round trip communications between the AEC and the authors. Artifacts can be submitted only in the AE time frame associated with the paper submission round.

In addition to accepted papers, papers that receive a major or minor revision decision are eligible for AE: at artifact submission time, their authors should justify the necessary changes that they intend to carry out on the initially submitted paper and how such changes relate to the submitted artifact.
In addition to accepted papers, papers that receive a major or minor revision decision are eligible for AE: at artifact submission time, their authors should justify the necessary changes that they intend to carry out on the initially submitted paper and how such changes relate to the submitted artifact.

At submission time, authors choose which [*badges*](badges) they want to be evaluated for. Members of the AEC will evaluate each artifact using the artifact appendix and instructions as guides, as detailed later in this page. Evaluators will communicate anonymously with authors through HotCRP to resolve minor issues and ask clarifying questions.

Expand Down
2 changes: 1 addition & 1 deletion _conferences/ndss2025/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,6 @@ order: 0

The AE process promotes the reproducibility of experimental results and the dissemination of artifacts to benefit our community as a whole. Publishing an artifact benefits, among others, how easily peers can build on it, use it as a comparison point, or solve questions about cases not considered by the original authors.

Authors of NDSS papers have the option of submitting their artifacts shortly after the notification of the (conditional) acceptance of their papers. Papers that pass artifact evaluation may include an additional 1-2 page appendix detailing the artifact and will have evaluation [badges](badges) on their first page.
Authors of NDSS papers have the option of submitting their artifacts shortly after the notification of the (conditional) acceptance of their papers. Papers that pass artifact evaluation will include a 2-page appendix detailing the artifact and have evaluation [badges](badges) on their first page.

The AE process recognizes authors who devote effort to make their work reusable and reproducible by others. This includes making artifacts publicly available, documenting and packaging their work in a way that facilitates reuse, and structuring experiments such that they can be repeated and the results reproduced by other researchers. The AEC will consider outstanding artifacts for Distinguished Artifact Awards.
4 changes: 2 additions & 2 deletions _conferences/ndss2025/organizers.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ title: Organizers
order: 30
---

Artifact Evaluation Co-chairs: [Daniele Cono D'Elia](https://www.diag.uniroma1.it/~delia/) (Sapienza University of Rome) and [Mathy Vanhoef](https://www.mathyvanhoef.com/)
Artifact Evaluation Co-chairs: [Daniele Cono D'Elia](https://www.diag.uniroma1.it/~delia/) (Sapienza University of Rome) and [Mathy Vanhoef](https://www.mathyvanhoef.com/) (KU Leuven).

### Artifact Evaluation Committee

TBA
The list of members is currently available on [HotCRP](https://ndss25ae-summer.hotcrp.com/users/pc) and will be featured below soon.

0 comments on commit fd4ba4c

Please sign in to comment.