Merge branch 'JOSS-Submission'

.github/workflows/JOSS_Paper.yaml

@@ -0,0 +1,23 @@
+on: [push]
+
+jobs:
+  paper:
+    runs-on: ubuntu-latest
+    name: Paper Draft
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Build draft PDF
+        uses: openjournals/openjournals-draft-action@master
+        with:
+          journal: joss
+          # This should be the path to the paper within your repo.
+          paper-path: paper.md
+      - name: Upload
+        uses: actions/upload-artifact@v1
+        with:
+          name: paper
+          # This is the output path where Pandoc will write the compiled
+          # PDF. Note, this should be the same directory as the input
+          # paper.md
+          path: paper.pdf

.github/workflows/documentation.yaml

@@ -31,8 +31,6 @@ jobs:
         cp -r Custom/* gh-pages/Custom
         mkdir -p  gh-pages/Images
         cp -r Images/* gh-pages/Images
-        mkdir -p  gh-pages/TCA
-        cp -r TCA/* gh-pages/TCA
         mkdir -p  gh-pages/lib
         cp -r lib/* gh-pages/lib
         cd gh-pages

Custom/utility.js

@@ -61,6 +61,7 @@ function removeEntity(){
     }
     if(els.length == 0){
         utility.checked = false;
+        selectedEntity = null;
         toggleAddEdit(true);
 
     } else {
@@ -350,6 +351,7 @@ function revertChanges(){
         bullseyeNum = 0;
         textureNum = 0;
         numAdded = 0;
+        selectedEntity = null;
 }
 
 names = {'default' : {'red':1,'green':1,'blue':1,'white':1,'grille':1,'crosshair':1,'line':1,'circular dot array':1,'dot array':1,'checkerboard (w)':1,'checkerboard (b)':1,'ring_w5':1,'ring_w10':1,'ring_w20':1,'bullseye':1}}

README.md

@@ -1,15 +1,18 @@
 # WebXR Tools
-This page contains a few tools and concepts for the use of WebVR in regulatory science.
+We have developed a regulatory science tool that enables the measurement of head-mounted display (HMD) quality by creating customized scenes and patterns. This tool in particular provides an easy way to display scenes on multiple headsets so that users can assess and compare image quality. This is accomplished through the use of known standardized patterns like the grille or dot array and simple entities like circles, rectangles, or triangles. This flexibility allows for the conduction of targeted examinations of specific display characteristics. For instance, users can create and share packages that are designed to assess display resolution, chromatic aberrations, color mapping, and other spatiotemporal effects. This tool also addresses the issue of pattern portability. Currently, HMDs are very heterogeneous, making it difficult to deploy the same pattern to multiple headsets. This barrier often comes up in standards meetings as an impeding force on the development and validation of different HMDs. In short, this tool provides scientists, medical device manufacturers, and regulators access to a centralized scene to test various display devices and provide reports that are easier to understand and, most importantly, replicate.
 
-The two notable tools are the [Custom Pattern Creator tool](./Custom) and the [TCA Test tool](./TCA)
+The Pattern Creation Tool consists of a user interface built using A-Frame, Three.js, and WebXR for the easy creation of image quality test patterns. It is housed on a website that can be accessed on any WebXR-compatible device with internet access. Created test patterns and collections of scenes can be downloaded and shared as JSON files. This feature enables users to easily create and share patterns, allowing the testing of the same scene on multiple devices. All uploaded patterns remain fully editable, allowing users to make necessary changes to patterns that are shared with them. Additionally, test patterns can be saved in groups or scenes to allow for testing multiple different patterns or variations of a pattern.
 
-The other two tools provided were proof of concepts and can be used as models for anyone wishing to do something similar.
+The tool is available here: [Custom Pattern Creator tool](https://didsr.github.io/WebXR-tools/Custom/)
+
+Documentation for the tool is available here: [Custom Pattern Creator tool](https://didsr.github.io/WebXR-tools/Custom/doc)
 
 
 ## Custom Pattern Creator
 
 ![plot](Images/patternGenerator.PNG)
 
-## TCA Test
+## Local Installation
+
+To install and use this tool locally, download the contents of the package from github and run index.html. If the tool is being run without connection to the internet, the link based sharing will not work. Instead, opt to manually download and upload files.
 
-![plot](Images/tca.PNG)

doc/Makefile

@@ -3,8 +3,8 @@
 
 # You can set these variables from the command line, and also
 # from the environment for the first two.
-SPHINXOPTS    ?=
-SPHINXBUILD   ?= sphinx-build
+SPHINXOPTS    +=
+SPHINXBUILD   += sphinx-build
 SOURCEDIR     = .
 BUILDDIR      = _build
 

doc/contributing.rst

@@ -0,0 +1,13 @@
+Contributing
+===================
+We encourage users to contribute to the tool by asking questions and reporting bugs.
+
+Asking a question
+-----------------------------
+Questions can be posted to `GitHub Issues <https://github.com/DIDSR/WebXR-tools/issues>`_ with the "Question" tag. Before posting a question, please ensure that no one else has asked this question. 
+
+
+Reporting bugs
+-----------------------------
+Bugs can be reported using `GitHub Issues <https://github.com/DIDSR/WebXR-tools/issues>`_. Please describe your bug in good detail. If possible, please provide a screenshot of the developer console. These issues will be addressed in a timely manner.
+

doc/index.rst

@@ -16,6 +16,7 @@ The Pattern Generation Tool provides a user interface for creating, editing, and
    patterns
    entities
    extra
+   contributing
 
 
 

doc/introduction.rst

@@ -1,7 +1,10 @@
 Introduction
 ================
 
-`Access to Pattern Creation Tool <https://brendanc490.github.io/WebVR-Test-Patterns/Custom/>`_
+`Access to Pattern Creation Tool <https://didsr.github.io/WebXR-tools/Custom/>`_
+-----------------------------------------------------------------------------------------------
+
+`View the source code <https://github.com/DIDSR/WebXR-tools/tree/main/Custom>`_
 -----------------------------------------------------------------------------------------------
 
 Features
@@ -34,6 +37,6 @@ Demo
 
 
 
-           <video controls src="_static/demoVid.webm" width="620" height="620">
+           <video controls src="./_static/demoVid.mp4" width="620" height="620">
 
-           </video>  
+           </video>

index.html

@@ -9,16 +9,16 @@
 <body>
     <h1>WebXR Test Patterns by DIDSR</h1>
     <ul id="menu">
-        <li>
+        <!--<li>
             <a href="./TCA/"><img src="Images/tca-menu.jpg">
                 <div class="title">TCA Test</div>
                 <div class="description">Test for Transverse Chromatic Aberration in immersive mode.</div>
             </a>
             <a class="instructions" href="https://github.com/DIDSR/WebXR-tools/tree/main/TCA"
                 title="See instructions">?</a>
-        </li>
+        </li>-->
         <li>
-            <a href="./Custom/"><img src="Images/patternGenerator.PNG">
+            <a href="./Custom/index.html"><img src="Images/patternGenerator.PNG">
                 <div class="title">Custom Pattern Creator</div>
                 <div class="description">Tool that allows for creation of patterns on WebXR. Scenes can be
                     saved and shared.
@@ -28,7 +28,7 @@ <h1>WebXR Test Patterns by DIDSR</h1>
                 title="See instructions">?</a>
         </li>
         <li>
-            <a href="./Compatibility/"><img src="Images/compatibility_in_action.png">
+            <a href="./Compatibility/index.html"><img src="Images/compatibility_in_action.png">
                 <div class="title">Compatibility test</div>
                 <div class="description">Tool that allows to test the compatibility of an XR device with WebXR.
                 </div>
@@ -42,4 +42,4 @@ <h1>WebXR Test Patterns by DIDSR</h1>
             href="https://github.com/DIDSR/WebXR-tools">See on GitHub</a>.</div>
 </body>
 
-</html>
+</html>

paper.bib

@@ -0,0 +1,33 @@
+@article{Beams:2020,
+	Author = {{Beams}, R. and {Collins}, B. and {Kim}, A. S. and {Badano}, A.},
+  	Doi = {10.1109/VR46266.2020.00108},
+	Journal = {2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)},
+	Keywords = {Virtual Reality, Image Quality},
+	Month = mar,
+	Title = {Angular Dependence of the Spatial Resolution in Virtual Reality Displays},
+	Year = 2020}
+
+@article{Zhao:2023,
+	Author = {{Zhao}, C. and {Beams}, R. and {Badano}, A.},
+  	Doi = {10.1002/jsid.1208},
+	Journal = {J Soc Inf Display},
+	Keywords = {Virtual Reality, Image Quality},
+	Month = May,
+	Title = {Radially variant contrast measurement in virtual reality headsets using circular concentric ring patterns.},
+	Year = 2023}
+
+
+@misc{AFrame,
+    Author        = {AFrame},
+    howpublished = {\url{https://aframe.io/}}
+}
+
+@misc{Three.js,
+    Author        = {Three.js},
+    howpublished = {\url{https://threejs.org/}}
+}
+
+@misc{WebXR,
+    Author        = {WebXR},
+    howpublished = {\url{https://immersiveweb.dev/}}
+}

paper.md

@@ -0,0 +1,62 @@
+---
+title: 'Open-Source Pattern Creation Tool for Medical Extended Reality Image Quality Assessment'
+tags:
+  - Javascript
+  - HTML
+  - CSS
+  - WebXR
+  - A-Frame
+  - Virtual Reality
+  - VR
+  - Image Quality
+  - Pattern Creation
+authors:
+  - name: Brendan Collins
+    orcid: 0000-0002-1529-5900
+    affiliation: "1, 2, 3"
+  - name: Ryan Beams
+    orcid: 0000-0003-2592-6064
+    affiliation: 1
+  - name: Miguel Lago
+    orcid: 0000-0002-7829-9570
+    affiliation: 1
+affiliations:
+ - name: Center for Devices and Radiological Health, U.S. Food and Drug Administration
+   index: 1
+ - name: ORISE Fellow, Oak Ridge Institute for Science and Research
+   index: 2
+ - name: Department of Computer Science, University of Maryland - College Park
+   index: 3
+date: 11 August 2023
+bibliography: paper.bib
+
+---
+
+# Summary
+
+The potential for mixed-reality devices in medicine has recently grown. In order to approve these devices for medical use, they must be evaluated on many characteristics to ensure safety and effectiveness. One of the most important evaluations is the quality of the device’s display, known as image quality, especially for medical applications [@Zhao:2023]. The image quality evaluation of virtual and augmented reality devices requires the creation of a variety of different test patterns. These test patterns consist of many different shapes that must frequently change their location, size, or color in order to adapt to the many headsets in the market [@Beams:2020]. The Pattern Creation Tool allows for the creation, editing, and sharing of patterns that work in a mixed reality environment. The Pattern Creation Tool includes multiple default patterns that are commonly used in image quality evaluation. These include a pattern for finding the center of the device’s display, a pattern to measure geometric distortion, a pattern to measure Michelson contrast, and a pattern to measure line spread. These patterns can make simple testing easier as well as serve as a template for other patterns.
+
+# Statement of need
+
+We have developed a regulatory science tool that enables the measurement of head-mounted display (HMD) quality by creating customized scenes and patterns. This tool in particular provides an easy way to display scenes on multiple headsets so that users can assess and compare image quality. This is accomplished through the use of known standardized patterns like the grille or dot array and simple entities like circles, rectangles, or triangles. Figure 1 demonstrates how to edit a pattern containing multiple of these entities. This flexibility allows for the conduction of targeted examinations of specific display characteristics. For instance, users can create and share packages that are designed to assess display resolution, chromatic aberrations, color mapping, and other spatiotemporal effects. This tool also addresses the issue of pattern portability. Currently, HMDs are very heterogeneous, making it difficult to deploy the same pattern to multiple headsets. This barrier often comes up in standards meetings as an impeding force on the development and validation of different HMDs. In short, this tool provides scientists, medical device manufacturers, and regulators access to a centralized scene to test various display devices and provide reports that are easier to understand and, most importantly, replicate.
+
+The Pattern Creation Tool consists of a user interface built using A-Frame [@AFrame], Three.js [@Three.js], and WebXR [@WebXR] for the easy creation of image quality test patterns. It is housed on a website that can be accessed on any WebXR-compatible device with internet access. Created test patterns and collections of scenes can be downloaded and shared as JSON files. This feature enables users to easily create and share patterns, allowing the testing of the same scene on multiple devices. All uploaded patterns remain fully editable, allowing users to make necessary changes to patterns that are shared with them. Additionally, test patterns can be saved in groups or scenes to allow for testing multiple different patterns or variations of a pattern. Figure 2 shows the display of a pattern that contains uploaded image textures.
+
+Traditionally, most mixed-reality development is done using video game engines such as Unity or Unreal Engine. These game engines have steep learning curves, need programming skills, and are also heterogeneous. As a result, it can be cumbersome to generate simple patterns for a wide variety of devices in a timely manner. The Pattern Creation Tool simplifies this process by creating a user interface to handle pattern creation (add entities, upload textures, etc.). This mitigates programming experience as a limiting factor. A second issue present with game engines like Unity and Unreal Engine is that it can be difficult to run projects on different devices. This is because device compatibility for mixed-reality devices is not standardized yet, so different devices may or may not be compatible on different platforms. The Pattern Creation Tool is available on the internet, so any device that is compatible with WebXR will be able to access the tool. This includes Oculus/Meta devices and Microsoft Extended Reality devices. This tool utilizes WebXR and A-Frame to ensure consistent and reliable results across devices. Lastly, game engines are usually not open source, which might cause further hurdles for research. Both WebXR and the Pattern Creation Tool are open-source and easy to use in research projects. The Pattern Creation tool has been used in a scientific publication, where the generated patterns aided the creation of a new methodology for the Image Quality analysis of virtual reality head-mounted displays [@Zhao:2023].
+
+# Figures
+
+![Editing a Pattern](./Images/Fig1a.png){ width=75% }  
+
+
+![Displaying a customized pattern](./Images/Fig1b.png){ width=75% }  
+
+
+# Acknowledgements
+
+Brendan Collins was supported by an appointment to the Research Participation Program at the Center for Devices and Radiological Health administered by the Oak Ridge Institute for Science and Education through an interagency agreement between the U.S. Department of Energy and the U.S. Food and Drug Administration. 
+
+# Disclaimer
+The mention of commercial products, their resources, or their use in connection with material reported herein is not to be construed as either an actual or implied endorsement of such products by the Department of Health and Human Services. This is a contribution of the U.S. Food and Drug Administration and not subject to copyright
+
+# References