diff --git a/.github/workflows/JOSS_Paper.yaml b/.github/workflows/JOSS_Paper.yaml new file mode 100644 index 0000000..971cce1 --- /dev/null +++ b/.github/workflows/JOSS_Paper.yaml @@ -0,0 +1,23 @@ +on: [push] + +jobs: + paper: + runs-on: ubuntu-latest + name: Paper Draft + steps: + - name: Checkout + uses: actions/checkout@v3 + - name: Build draft PDF + uses: openjournals/openjournals-draft-action@master + with: + journal: joss + # This should be the path to the paper within your repo. + paper-path: paper.md + - name: Upload + uses: actions/upload-artifact@v1 + with: + name: paper + # This is the output path where Pandoc will write the compiled + # PDF. Note, this should be the same directory as the input + # paper.md + path: paper.pdf diff --git a/.github/workflows/documentation.yaml b/.github/workflows/documentation.yaml index 5711e69..09637ac 100644 --- a/.github/workflows/documentation.yaml +++ b/.github/workflows/documentation.yaml @@ -31,8 +31,6 @@ jobs: cp -r Custom/* gh-pages/Custom mkdir -p gh-pages/Images cp -r Images/* gh-pages/Images - mkdir -p gh-pages/TCA - cp -r TCA/* gh-pages/TCA mkdir -p gh-pages/lib cp -r lib/* gh-pages/lib cd gh-pages diff --git a/Custom/README.md b/Custom/README.md index 627cbfc..cdec24c 100644 --- a/Custom/README.md +++ b/Custom/README.md @@ -1,245 +1,168 @@ -# Pattern Generation Tool # -The Pattern Generation Tool provides a user interface for creating, editing, and displaying [WebXR](https://immersiveweb.dev/) scenes. -Scenes consist of 2 and 3 dimensional entities taken from [A-Frame](https://aframe.io/docs/1.4.0/core/entity.html). -These entities can be manipulated in a variety of different ways to create flexible patterns. -Scenes are grouped together in packages that can be downloaded and shared. +# Manual User Interface Tests # + +Begin by opening this link: https://didsr.github.io/WebXR-tools/Custom/ + +## Test 1- Adding and editing a package +This test will examine the package and pattern systems. + 1. Add a package with the name "Test 1" by pressing the *plus* icon next to **Packages** + 2. Add three patterns to this package by pressing the *plus* icon next to **Pattern List** + - Name the first pattern "Pattern 1" + - Name the second pattern "Pattern 2" + - Name the third pattern "Pattern 3" + + At this point, the tool should look like this: + ![image](../Images/test1_1.PNG) + + 3. Reorder the patterns so that "Pattern 3" is the first pattern, "Pattern 2" is the second pattern, and "Pattern 1" is the last pattern + - Do this by dragging "Pattern 3" and dropping it in the first position + - Drag "Pattern 2" and drop it in the second position + 4. Switch packages to the "default" package + 5. Multi-select the patterns "crosshair" and "ring_w2" by holding Ctrl while clicking on the patterns + 6. Copy these patterns by pressing Ctrl+C on windows or Cmd+C on Mac + 7. Switch packages to the "Test 1 Package" package + 8. Paste the copied patterns by pressing Ctrl+V or Cmd+V + + At this point, the tool should look like this: + ![image](../Images/test1_2.PNG) + +## Test 2- Adding and editing a pattern +This test will examine the pattern and entity systems. + 9. Rename the "crosshair" pattern to "Pattern 4" + - Select the "crosshair" pattern by clicking on it + - Open settings by pressing the *gear* icon at the top right of the tab + - Under **Pattern Settings** select the *Rename* button + - Enter "Pattern 4" in the textbox and press OK + 10. Delete the "ring_w2" pattern by pressing the *trash* icon next to **Pattern List** + - Select OK on the confirmation + 11. Close the settings by pressing the *X* icon at the top right of the tab + 11. Reorder the patterns so that "Pattern 4" is first, "Pattern 3" is second, "Pattern 2" is third, and "Pattern 1" is last + + At this point, the tool should look like this: + ![image](../Images/test1_3.PNG) + + 12. Select "Pattern 3" + 13. Switch the toggle at the top of the tab to be in *Edit* mode + 14. Add a circle, plane, and triangle to this pattern + - Select circle as the type of entity to add from the dropdown next to **Type of entity:** + - Press the *Add to Pattern* button to add it to the pattern + - Repeat for plane and triangle + - The entities will spawn at random places on the screen + 15. Switch the second toggle to *Edit Entity* mode + 16. Edit the Circle + - Click on the circle or select "circle0" from the **Entity** dropdown + - Move the circle so that it is in position (0, 0, 250) + - Change the color of the circle to white by selecting it from the color selector or entering #ffffff + - Increase the radius of the circle to 40m + - Decrease the border size to 15m + 17. Edit the Plane + - Click on the plane or select "plane0" from the **Entity** dropdown + - Move the plane so that it is in position (-20, 100, 250) + - Change the color of the plane to blue by selecting it from the color selector or entering #0000ff + - Increase the height of the plane to 100m + - Decrease the width of the plane to 20m + - If desired, add a texture from the drop down or upload an image file to use as a texture + - If an uploaded texture is added here, one step will change in Part 3 + - Adding a texture will rescale the entity to meet the aspect ratio of the image being used + 18. Edit the Triangle + - Click on the triangle or select "triangle0" from the **Entity** dropdown + - Move the triangle so that it is in position (20, -100, 250) + - Change the color of the triangle to green by selecting it from the color selector or entering #00ff00 + - Change vertex A to be at (0, 50) + - Change vertex B to be at (-50, -50) + - Change vertex C to be at (50, -50) + At this point the tool should look like this: + + ![image](../Images/test2_1.PNG) + + 19. Switch the first toggle to *Display* mode + 20. Select "Pattern 2" + 21. Switch the toggle at the top of the tab to be in *Edit* mode + 22. Add a gradient, checkerboard, and grille to this pattern + - Select gradient as the type of entity to add from the dropdown next to **Type of entity:** + - Press the *Add to Pattern* button to add it to the pattern + - Repeat for checkerboard and grille + - The entities will spawn at random places on the screen + 23. Switch the second toggle to *Edit Entity* mode + 24. Edit the Gradient + - Click on the gradient or select "gradient0" from the **Entity** dropdown + - Move the gradient so that it is in position (0, 150, 250) + - Change the primary color of the gradient to white by selecting it from the color selector or entering #ffffff + - Increase the bar height to 20m + - Decrease the bar width to 3m + - Increase the number of bars to 100 + - Change the secondary color to red by selecting it from the color selector or entering #ff0000 + 25. Edit the Checkerboard + - Click on the checkerboard or select "checkerboard0" from the **Entity** dropdown + - Move the plane so that it is in position (0, 0, 250) + - Change the primary color of the checkerboard to purple by selecting it from the color selector or entering #800080 + - Increase the number of columns to 20 + - Increase the number of rows to 19 + - Increase the size of the tiles to 10m + - Change the secondary color to white by selecting it from the color selector or entering #ffffff + 26. Edit the grille + - Click on the grille or select "grille0" from the **Entity** dropdown + - Move the gradient so that it is in position (0, -150, 250) + - Change the primary color of the grille to orange by selecting it from the color selector or entering #ffa500 + - Decrease the bar height to 20m + - Decrease the bar width to 3m + - Increase the number of bars to 100 + - Change the secondary color to white by selecting it from the color selector or entering #ffffff + At this point the tool should look like this: +![image](../Images/test2_2.png) + + 27. Switch the first toggle to *Display* mode + 28. Select "Pattern 1" + 29. Switch the toggle at the top of the tab to be in *Edit* mode + 30. Add a dot array, circular dot array, and bullseye to this pattern + - Select dot array as the type of entity to add from the dropdown next to **Type of entity:** + - Press the *Add to Pattern* button to add it to the pattern + - Repeat for circular dot array and bullseye + - The entities will spawn at random places on the screen + 31. Switch the second toggle to *Edit Entity* mode + 32. Edit the Dot Array + - Click on the dot array or select "dotarray0" from the **Entity** dropdown + - Move the dot array so that it is in position (-20, -100, 250) + - Change the color of the dot array to green by selecting it from the color selector or entering #00ff00 + - Increase the number of columns to 7 + - Increase the number of rows to 7 + - Increase the radius of the dots to 3m + - Decrease the spacing of the dots to 5m + - Toggle on the center dot being filled + 33. Edit the Circular Dot Array + - Click on the circular dot array or select "circularDotarray0" from the **Entity** dropdown + - Move the circular dot array so that it is in position (0, 0, 249) + - Change the color of the dot array to white by selecting it from the color selector or entering #ffffff + - Increase the number of dots to 15 + - Increase the number of rings to 6 + - Keep the radius of the dots at 2m + - Keep the spacing of the dots at 10m + - Keep the center dot hollow + 34. Edit the bullseye + - Click on the bullseye or select "bullseye0" from the **Entity** dropdown + - Move the bullseye so that it is in position (0, 0, 250) + - Change the primary color of the bullseye to blue by selecting it from the color selector or entering #0000ff + - Increase the number of rings to 6 + - Keep the ring pitch at 5m +At this point the tool should look like this: +![image](../Images/test2_3.PNG) + +## Test 3- Saving and uploading a package +This test will examine the sharing feature of the tool. + + 35. Switch the first toggle to *Display* mode + 36. Select the save package button next to **Packages:** + - Note: if an uploaded texture was used in Test 2, select cancel when prompted to include textures + - Uploaded textures cause generated files to be too large for the link generation service + 37. Make a note of the package id saved in the url after "?id=" ('?id=') + 38. Refresh the page + 39. Ensure that the package "Test 1" is loaded and the patterns are correct + 40. Upload the "Test 1 Comparison" package + - Open settings by hitting the *gear* icon at the top right of the tab + - Press the import button denoted by a cloud with a down arrow + - Paste this link into the textbox: https://didsr.github.io/WebXR-tools/Custom/?id=598zgfm3bqw9 + 41. Check that there is now a package called "Test 1 Comparison" with the same patterns as those created in these tests + 42. To interactively check for correct results, navigate between the two packages and ensure entity values are the same + 43. Close the tab completely + 44. Navigate back to the starting page of the tool (https://didsr.github.io/WebXR-tools/Custom/) + 45. Select "Test 1" and "Test 1 Comparison" from the **Recent Packages** dropdown and ensure that they are correct -### Features -- Create packages of scenes that can be shared via a file or a link. -- Order of scenes within the package can be easily changed. -- Scenes are fully editable upon uploading or downloading. -- Scenes can be switched between using arrow keys or when in immersive mode, the buttons on the left and right controllers. -- Default package available containing commonly used patterns: - - Red: A solid red background - - Green: A solid green background - - Blue: A solid blue background - - White: A solid white background - - Grille: A black and white background - - Crosshair: A white crosshair - - Line: A white vertical line - - Dot Array: A matrix of dots - - Circular Dot Array: A dot array arranged in a circle - - Checkerboard_w: A checkerboard with a white center tile - - Checkerboard_b: A checkerboard with a black center tile - - Ring_w1: A bullseye with a high pitch - - Ring_w2: A bullseye with a medium pitch - - Ring_w5: A bullseye with a low pitch - - -# Directions For Use # -### Packages -1. [Create a Package](#createPackage) -2. Edit a Package -3. Display a Package -4. Save a Package -5. Share a Package - -### Patterns -1. Create a Pattern -2. Edit a Pattern -3. Display a Pattern -4. Save a Pattern -5. Share a Pattern - - -
-## Create a Package - -## Edit a Package - -## Display a Package - -## Save a Package - -## Share a Package - - - - - - - - - -## Displaying Patterns - - - -### Selecting a Pattern -To select a pattern, first ensure that the checkbox for the pattern under ***Pattern List*** has been checked. Then select the desired pattern from the ***Pattern Display*** dropdown. - -### Adding a Pattern -A pattern a can be added using the ***Add Pattern*** button. A name for the added pattern can be specified using textbox above the ***Add Pattern*** button. - -### Removing a Pattern -Using the ***Remove Pattern*** button, the currently displayed pattern can be removed. - -### Selecting Multiple Patterns -To select multiple patterns, check all of the desired checkboxes under ***Pattern List***. -**The order that the patterns are selected in will be the display order** -The current pattern can be rotated through using the up or down arrows on the keyboard or the buttons on the left and right controllers. -Any buttons on the left controller step back one pattern and any buttons on the right controller step forward one pattern. - -### Displaying the Desired Pattern -To display the pattern, hit the ***VR*** button in the bottom right hand corner. -[**Check the demos folder for some basic editable patterns**](./demos) - -### Saving and Uploading a Group of Patterns -Multiple patterns can be saved together by hitting the ***Save Selected Patterns*** button. This will save all currently selected patterns from the ***Pattern List*** as a JSON file. -To upload a group of patterns, press the ***Upload Patterns*** button and select the desired JSON file. This JSON file can contain one or more patterns and will add them all to the ***Pattern List***. - -## Editing Patterns - -### Uploading a Pattern -A JSON file containing a single pattern can be uploaded to the ***Upload a pattern*** input. If a file containing multiple patterns is uploaded, it will be rejected. - -### Uploading images -Image files are treated as entity textures. Here are the steps on how to properly upload these files. - -1. Add a plane to a scene. Planes are the only entities that can be textured. -2. Click the edit entity toggle and locate the input for textures labeled "Upload your textures here." Any number of files can be uploaded here. Images uploaded will be available as textures in the dropdown labeled "Texture". -3. Select the desired texture from the "Texture" dropdown. Selecting a texture will automatically resize the plane to the images aspect ratio. To retain the original image colors, set the plane color to be white. - -### Adding Entities -![plot](../Images/custom.PNG) -To add an entity, make sure the slider at the top left of the webpage is set to ***Edit Pattern*** then move the slider below it to the ***Add Entity***. - -Then, select the type of entity you would like to add using the ***Type of entity*** dropdown and hit the ***Add to Pattern*** button. - -Entities will spawn at a random position within the editor camera field of view with a random color. - -***Entities may appear skewed in the editor but this affect is corrected on the headset. Do not stack entities, it will result in [z-fighting].(https://en.wikipedia.org/wiki/Z-fighting)*** - -#### Types of Entities -- Circle -- Plane -- Triangle -- Gradient -- Checkerboard -- Grille - -### Editing Entities -![plot](../Images/editorExample.PNG) -**In order to access the editor one or more entities must be added to the pattern**. The editing bar can be hidden by pressing the ***Hide Settings*** button. - -Once entities have been added to the pattern, change the slider to ***Edit Entity***. - -To change the entity being edited, either click on the entity or select the desired entity from the dropdown labeled ***Current Entity***. -Entity ID's are automatically generated with the format **\\**. - -Examples: circle0, checkerboard4, plane20. - -The current entity can be removed by hitting the ***Remove from Pattern***. The entire pattern can be reset by hitting the ***Reset Pattern*** button. - -### Saving the Pattern -To save the current pattern, press the ***Save Pattern*** button located at the bottom of the editor tab. The pattern is saved as an editable JSON file. Uploaded textures are preserved without image compression. - -### Background Color -To change the background color, select a new color or enter the hexadecimal code of the desired color. - -### Universal Entity Settings -#### Position: (x deg, y world units) -To edit the position, locate the position text boxes under **Universal Entity Settings**. The left box refers to the x position and the y position. - -The position of an entity refers to where its center point is located. The editor interfaces the three axis in different manners. -Positional data is calculated based on a cylinder that encompasses the camera. The radius of this cylinder cannot be changed, however the cylinder has an infinite height. The x and z axes are responsible for horizontal positioning. The y axis controls the vertical position. - -![plot](../Images/cylinderRadius.PNG) - -- x-axis: - - This axis is in terms of **degrees** where 1 degree references 1 degree of clockwise rotation away from the fixed camera center on the cylinder. -- y-axis: - - This axis is in terms of **world units** where 1 world unit is equal to 1 meter in 3D space. -- z-axis: - - **The z-axis is NOT editable by users.** The location on the z-axis is automatically calculated based on the x-axis angle. - -The reason that the radius of the cylinder is not editable is due to the fact that all entities are 2-dimensional. Changing their depth would essentially have the same effect as making entities smaller or larger. - -#### Rotation: θ deg -To edit the rotation, locate the text box labeled ***Rotation*** under **Universal Entity Settings**. - -For this program, the rotation of an entity refers to rotation around the **z-axis only**. The axis of rotation is located at the center point of entity. The unit for rotation is **degrees** where 1 degree refers to one degree of rotation about the axis of rotation. - -#### Color: #HEXCODE -To edit the color, locate the color selector labeled ***Color*** under **Universal Entity Settings**. - -Select a color using the color slider or input the desired color in hexadecimal format. - -***For checkerboards and grilles, the alternate color will always be black*** - -### Entity Specific Settings -#### Circle -- Radius (m): - - The unit for radius is world units where 1 world unit is equal to 1 meter in 3D space. -- Border Size: - - The unit for border size is world units where 1 world unit is equal to 1 meter in 3D space. By default, the border size is set to be equal to the radius of the circle, meaning the entity is entirely filled in. - -#### Plane -- Texture: - - There are some textures built into the site. [They can be found here.](./textures) Textures can also be uploaded as JPGS or PNGS and added to the list of available textures in the pattern. - - Applying a texture will automatically scale the plane to fit the aspect ratio. -- Height (m): - - The unit for height is world units where 1 world unit is equal to 1 meter in 3D space. -- Width (m): - - The unit for width is world units where 1 world unit is equal to 1 meter in 3D space. -- Border Size: - - The unit for border size is world units where 1 world unit is equal to 1 meter in 3D space. By default, the border size is set to be equal to the radius to the smallest dimension present of the plane, meaning the entity is entirely filled in. - -#### Triangle -![plot](../Images/triangle.PNG) -- Vertex A (x m,y m): - - Position of vertex A in relation to the center point of the entity. The coordinates are in world units where 1 world unit is equal to 1 meter in 3D space. -- Vertex B (x m,y m): - - Position of vertex B in relation to the center point of the entity. The coordinates are in world units where 1 world unit is equal to 1 meter in 3D space. -- Vertex C (x m,y m): - - Position of vertex C in relation to the center point of the entity. The coordinates are in world units where 1 world unit is equal to 1 meter in 3D space. - -#### Gradient -- Individual Bar Height (m): - - The unit for height is world units where 1 world unit is equal to 1 meter in 3D space. -- Indivdual Bar Width (m): - - The unit for width is world units where 1 world unit is equal to 1 meter in 3D space. -- Number of Bars: - - The number of bars in the gradient, the default is 32. - - The color is divided by the number of bars to determine the color of each bar. - -#### Checkerboard -- Tile size (m): - - The unit for width and height of tiles is world units where 1 world unit is equal to 1 meter in 3D space. -- Number of Rows: - - The number of rows in the checkerboard, the default is 16. -- Number of Columns: - - The number of columns in the checkerboard, the default is 16. - -#### Grille -- Individual Bar Height (m): - - The unit for height is world units where 1 world unit is equal to 1 meter in 3D space. -- Indivdual Bar Width (m): - - The unit for width is world units where 1 world unit is equal to 1 meter in 3D space. -- Number of Bars: - - The number of bars in the grille, the default is 32. - -### Other Notes -If you would like to inspect the HTML or JavaScript data of the selected entity: -- Open the inspector with Ctrl + Shift + i -- Enter "selectedEntity" into the command prompt, this will bring up all associated HTML code -- Enter "selectedEntity.getAttribute("\") to see the associated JavaScript data for the desired component - - Ex: selectedEntity.getAttribute("position") to view position or selectedEntity.getAttribute("material") to view texture and color -- Enter "selectedEntity.setAttribute("\",\) to set the associated JavaScript data for the desired component - - Ex: selectedEntity.setAttribute("position",{x: 0, y: 0, z: -125}) to set position to 0,0 on the cylindrical plane - - Ex: selectedEntity.setAttribute("material",{color: "#0000FF", shader: "flat", src: ""}) to set the color to red and remove the texture, ***make sure to include shader: "flat" to remove unnecessary lighting effects*** - - -## JSON File Format -{ -  "scenes": -    { map of scenes }, -   "textures": { -     "uploadedTextureFormats": -            { map of image sizes for uploaded textures }, -     "textureValues": -            [ list of textures associated with patterns being saved], -   } -   "date": "datetime of save" -} - diff --git a/Custom/utility.js b/Custom/utility.js index dcf9bfb..d3e9430 100644 --- a/Custom/utility.js +++ b/Custom/utility.js @@ -61,6 +61,7 @@ function removeEntity(){ } if(els.length == 0){ utility.checked = false; + selectedEntity = null; toggleAddEdit(true); } else { @@ -350,6 +351,7 @@ function revertChanges(){ bullseyeNum = 0; textureNum = 0; numAdded = 0; + selectedEntity = null; } names = {'default' : {'red':1,'green':1,'blue':1,'white':1,'grille':1,'crosshair':1,'line':1,'circular dot array':1,'dot array':1,'checkerboard (w)':1,'checkerboard (b)':1,'ring_w5':1,'ring_w10':1,'ring_w20':1,'bullseye':1}} diff --git a/Images/test1_1.PNG b/Images/test1_1.PNG new file mode 100644 index 0000000..2d69319 Binary files /dev/null and b/Images/test1_1.PNG differ diff --git a/Images/test1_2.PNG b/Images/test1_2.PNG new file mode 100644 index 0000000..03cd495 Binary files /dev/null and b/Images/test1_2.PNG differ diff --git a/Images/test1_3.PNG b/Images/test1_3.PNG new file mode 100644 index 0000000..fa82e3d Binary files /dev/null and b/Images/test1_3.PNG differ diff --git a/Images/test2_1.PNG b/Images/test2_1.PNG new file mode 100644 index 0000000..f8a6310 Binary files /dev/null and b/Images/test2_1.PNG differ diff --git a/Images/test2_2.png b/Images/test2_2.png new file mode 100644 index 0000000..bd52cd2 Binary files /dev/null and b/Images/test2_2.png differ diff --git a/Images/test2_3.PNG b/Images/test2_3.PNG new file mode 100644 index 0000000..11e62e9 Binary files /dev/null and b/Images/test2_3.PNG differ diff --git a/README.md b/README.md index 49923a6..47621de 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,18 @@ # WebXR Tools -This page contains a few tools and concepts for the use of WebVR in regulatory science. +We have developed a regulatory science tool that enables the measurement of head-mounted display (HMD) quality by creating customized scenes and patterns. This tool in particular provides an easy way to display scenes on multiple headsets so that users can assess and compare image quality. This is accomplished through the use of known standardized patterns like the grille or dot array and simple entities like circles, rectangles, or triangles. This flexibility allows for the conduction of targeted examinations of specific display characteristics. For instance, users can create and share packages that are designed to assess display resolution, chromatic aberrations, color mapping, and other spatiotemporal effects. This tool also addresses the issue of pattern portability. Currently, HMDs are very heterogeneous, making it difficult to deploy the same pattern to multiple headsets. This barrier often comes up in standards meetings as an impeding force on the development and validation of different HMDs. In short, this tool provides scientists, medical device manufacturers, and regulators access to a centralized scene to test various display devices and provide reports that are easier to understand and, most importantly, replicate. -The two notable tools are the [Custom Pattern Creator tool](./Custom) and the [TCA Test tool](./TCA) +The Pattern Creation Tool consists of a user interface built using A-Frame, Three.js, and WebXR for the easy creation of image quality test patterns. It is housed on a website that can be accessed on any WebXR-compatible device with internet access. Created test patterns and collections of scenes can be downloaded and shared as JSON files. This feature enables users to easily create and share patterns, allowing the testing of the same scene on multiple devices. All uploaded patterns remain fully editable, allowing users to make necessary changes to patterns that are shared with them. Additionally, test patterns can be saved in groups or scenes to allow for testing multiple different patterns or variations of a pattern. -The other two tools provided were proof of concepts and can be used as models for anyone wishing to do something similar. +The tool is available here: [Custom Pattern Creator tool](https://didsr.github.io/WebXR-tools/Custom/) + +Documentation for the tool is available here: [Custom Pattern Creator tool](https://didsr.github.io/WebXR-tools/Custom/doc) ## Custom Pattern Creator ![plot](Images/patternGenerator.PNG) -## TCA Test +## Local Installation + +To install and use this tool locally, download the contents of the package from github and run index.html. If the tool is being run without connection to the internet, the link based sharing will not work. Instead, opt to manually download and upload files. -![plot](Images/tca.PNG) diff --git a/TCA/README.md b/TCA/README.md deleted file mode 100644 index 143b3a5..0000000 --- a/TCA/README.md +++ /dev/null @@ -1,61 +0,0 @@ -# TCA Test Instructions - -![plot](../Images/tca.PNG) - -## Setup -You can move the centering and change the offset of the bars from the center point while in VR mode also. -In addition, you can control the size of the step for these. The step starts at 1 degree by default and can be changed by +- 0.1 degree. -It is universal, so changing the step when in centering mode will also change the step size when in offset mode. This is all done on the left controller. - -Once you have found a good centering and offset, I recommend pressing the menu button on the left controller to exit VR mode and jotting these values down. These values will be stored in the center and offset text boxes. You can input these values when you open the website so you don't need to redo the centering/offset everytime. - -## Moving Bars -The program allows you to control the red, green, or blue segments in both of the two bars. **You can only move one segment of a single bar at a time**. This is done with the right controller. - -## Controls - -### Keyboard -- a: Move current segment left -- d: Move current segment right -- Arrow up: Swap current segment to be one above (Red -> Red; Green -> Red; Blue -> Green) -- Arrow down: Swap current segment to be one below (Red -> Green; Green -> Blue; Blue -> Blue) -- t: Swap between left and right bar (default: right) -- n: Move centering left/increase offset -- m: Move centering right/decrease offset -- x: Swap between centering and offset mode (default: centering) -- c: Increase step -- z: Decrease step -- Space: Show/hide centering dot -- esc: Exit immersive mode - -### Controllers -**Currently, the only controllers that are supported are the Oculus Rift, Rift S, Oculus Quest 1 and 2.** If you would like to add new controllers, [check here to see if the controllers are supported.](https://aframe.io/docs/1.3.0/introduction/interactions-and-controllers.html) If the controllers are supported replace the following code with whatever controller you would like to use. **Make sure to take note of what buttons are available on this controller.** - - -![plot](../Images/tcaControllerAdd.PNG) - -***Note: Some controllers have different button layouts which may result in needing to rebind controls. This can be done by adding in button listeners like the one below tailored to the new constraints. Each action on the controller is mapped to a key on the keyboard, so to change the button mapping for controllers, replace which key event is created on a button press.*** - -#### Left Controller Bindings -![plot](../Images/tcaControllerL.PNG) - - -#### Right Controller Bindings -![plot](../Images/tcaControllerR.PNG) - - - -#### Right Controller: -- a: Move current segment left -- b: Move current segment right -- trigger: Swap current segment to be one above -- grip: Swap current segment to the one below -- Press in thumbstick: Hide center dot - -#### Left Controller: -- x: Move centering left/increase offset -- y: Move centering right/decrease offset -- trigger: Increase step -- grip: Decrease step -- Press in thumbstick: Swap between centering and offset -- menu: Exit VR diff --git a/doc/Makefile b/doc/Makefile index d4bb2cb..304d8b3 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -3,8 +3,8 @@ # You can set these variables from the command line, and also # from the environment for the first two. -SPHINXOPTS ?= -SPHINXBUILD ?= sphinx-build +SPHINXOPTS += +SPHINXBUILD += sphinx-build SOURCEDIR = . BUILDDIR = _build diff --git a/doc/_static/demoVid.mp4 b/doc/_static/demoVid.mp4 new file mode 100644 index 0000000..0e9056b Binary files /dev/null and b/doc/_static/demoVid.mp4 differ diff --git a/doc/_static/demoVid.webm b/doc/_static/demoVid.webm deleted file mode 100644 index 5ac5660..0000000 Binary files a/doc/_static/demoVid.webm and /dev/null differ diff --git a/doc/contributing.rst b/doc/contributing.rst new file mode 100644 index 0000000..190cb8d --- /dev/null +++ b/doc/contributing.rst @@ -0,0 +1,13 @@ +Contributing +=================== +We encourage users to contribute to the tool by asking questions and reporting bugs. + +Asking a question +----------------------------- +Questions can be posted to `GitHub Issues `_ with the "Question" tag. Before posting a question, please ensure that no one else has asked this question. + + +Reporting bugs +----------------------------- +Bugs can be reported using `GitHub Issues `_. Please describe your bug in good detail. If possible, please provide a screenshot of the developer console. These issues will be addressed in a timely manner. + diff --git a/doc/index.rst b/doc/index.rst index 3159f80..e861539 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -16,6 +16,7 @@ The Pattern Generation Tool provides a user interface for creating, editing, and patterns entities extra + contributing diff --git a/doc/introduction.rst b/doc/introduction.rst index c0a2894..2fdba23 100644 --- a/doc/introduction.rst +++ b/doc/introduction.rst @@ -1,7 +1,10 @@ Introduction ================ -`Access to Pattern Creation Tool `_ +`Access to Pattern Creation Tool `_ +----------------------------------------------------------------------------------------------- + +`View the source code `_ ----------------------------------------------------------------------------------------------- Features @@ -34,6 +37,6 @@ Demo - diff --git a/index.html b/index.html index 4374513..22f8985 100644 --- a/index.html +++ b/index.html @@ -9,16 +9,16 @@

WebXR Test Patterns by DIDSR

- \ No newline at end of file + diff --git a/paper.bib b/paper.bib new file mode 100644 index 0000000..890a603 --- /dev/null +++ b/paper.bib @@ -0,0 +1,33 @@ +@article{Beams:2020, + Author = {{Beams}, R. and {Collins}, B. and {Kim}, A. S. and {Badano}, A.}, + Doi = {10.1109/VR46266.2020.00108}, + Journal = {2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)}, + Keywords = {Virtual Reality, Image Quality}, + Month = mar, + Title = {Angular Dependence of the Spatial Resolution in Virtual Reality Displays}, + Year = 2020} + +@article{Zhao:2023, + Author = {{Zhao}, C. and {Beams}, R. and {Badano}, A.}, + Doi = {10.1002/jsid.1208}, + Journal = {J Soc Inf Display}, + Keywords = {Virtual Reality, Image Quality}, + Month = May, + Title = {Radially variant contrast measurement in virtual reality headsets using circular concentric ring patterns.}, + Year = 2023} + + +@misc{AFrame, + Author = {AFrame}, + howpublished = {\url{https://aframe.io/}} +} + +@misc{Three.js, + Author = {Three.js}, + howpublished = {\url{https://threejs.org/}} +} + +@misc{WebXR, + Author = {WebXR}, + howpublished = {\url{https://immersiveweb.dev/}} +} diff --git a/paper.md b/paper.md new file mode 100644 index 0000000..0f339f4 --- /dev/null +++ b/paper.md @@ -0,0 +1,62 @@ +--- +title: 'Open-Source Pattern Creation Tool for Medical Extended Reality Image Quality Assessment' +tags: + - Javascript + - HTML + - CSS + - WebXR + - A-Frame + - Virtual Reality + - VR + - Image Quality + - Pattern Creation +authors: + - name: Brendan Collins + orcid: 0000-0002-1529-5900 + affiliation: "1, 2, 3" + - name: Ryan Beams + orcid: 0000-0003-2592-6064 + affiliation: 1 + - name: Miguel Lago + orcid: 0000-0002-7829-9570 + affiliation: 1 +affiliations: + - name: Center for Devices and Radiological Health, U.S. Food and Drug Administration + index: 1 + - name: ORISE Fellow, Oak Ridge Institute for Science and Research + index: 2 + - name: Department of Computer Science, University of Maryland - College Park + index: 3 +date: 11 August 2023 +bibliography: paper.bib + +--- + +# Summary + +The potential for mixed-reality devices in medicine has recently grown. In order to approve these devices for medical use, they must be evaluated on many characteristics to ensure safety and effectiveness. One of the most important evaluations is the quality of the device’s display, known as image quality, especially for medical applications [@Zhao:2023]. The image quality evaluation of virtual and augmented reality devices requires the creation of a variety of different test patterns. These test patterns consist of many different shapes that must frequently change their location, size, or color in order to adapt to the many headsets in the market [@Beams:2020]. The Pattern Creation Tool allows for the creation, editing, and sharing of patterns that work in a mixed reality environment. The Pattern Creation Tool includes multiple default patterns that are commonly used in image quality evaluation. These include a pattern for finding the center of the device’s display, a pattern to measure geometric distortion, a pattern to measure Michelson contrast, and a pattern to measure line spread. These patterns can make simple testing easier as well as serve as a template for other patterns. + +# Statement of need + +We have developed a regulatory science tool that enables the measurement of head-mounted display (HMD) quality by creating customized scenes and patterns. This tool in particular provides an easy way to display scenes on multiple headsets so that users can assess and compare image quality. This is accomplished through the use of known standardized patterns like the grille or dot array and simple entities like circles, rectangles, or triangles. Figure 1 demonstrates how to edit a pattern containing multiple of these entities. This flexibility allows for the conduction of targeted examinations of specific display characteristics. For instance, users can create and share packages that are designed to assess display resolution, chromatic aberrations, color mapping, and other spatiotemporal effects. This tool also addresses the issue of pattern portability. Currently, HMDs are very heterogeneous, making it difficult to deploy the same pattern to multiple headsets. This barrier often comes up in standards meetings as an impeding force on the development and validation of different HMDs. In short, this tool provides scientists, medical device manufacturers, and regulators access to a centralized scene to test various display devices and provide reports that are easier to understand and, most importantly, replicate. + +The Pattern Creation Tool consists of a user interface built using A-Frame [@AFrame], Three.js [@Three.js], and WebXR [@WebXR] for the easy creation of image quality test patterns. It is housed on a website that can be accessed on any WebXR-compatible device with internet access. Created test patterns and collections of scenes can be downloaded and shared as JSON files. This feature enables users to easily create and share patterns, allowing the testing of the same scene on multiple devices. All uploaded patterns remain fully editable, allowing users to make necessary changes to patterns that are shared with them. Additionally, test patterns can be saved in groups or scenes to allow for testing multiple different patterns or variations of a pattern. Figure 2 shows the display of a pattern that contains uploaded image textures. + +Traditionally, most mixed-reality development is done using video game engines such as Unity or Unreal Engine. These game engines have steep learning curves, need programming skills, and are also heterogeneous. As a result, it can be cumbersome to generate simple patterns for a wide variety of devices in a timely manner. The Pattern Creation Tool simplifies this process by creating a user interface to handle pattern creation (add entities, upload textures, etc.). This mitigates programming experience as a limiting factor. A second issue present with game engines like Unity and Unreal Engine is that it can be difficult to run projects on different devices. This is because device compatibility for mixed-reality devices is not standardized yet, so different devices may or may not be compatible on different platforms. The Pattern Creation Tool is available on the internet, so any device that is compatible with WebXR will be able to access the tool. This includes Oculus/Meta devices and Microsoft Extended Reality devices. This tool utilizes WebXR and A-Frame to ensure consistent and reliable results across devices. Lastly, game engines are usually not open source, which might cause further hurdles for research. Both WebXR and the Pattern Creation Tool are open-source and easy to use in research projects. The Pattern Creation tool has been used in a scientific publication, where the generated patterns aided the creation of a new methodology for the Image Quality analysis of virtual reality head-mounted displays [@Zhao:2023]. + +# Figures + +![Editing a Pattern](./Images/Fig1a.png){ width=75% } + + +![Displaying a customized pattern](./Images/Fig1b.png){ width=75% } + + +# Acknowledgements + +Brendan Collins was supported by an appointment to the Research Participation Program at the Center for Devices and Radiological Health administered by the Oak Ridge Institute for Science and Education through an interagency agreement between the U.S. Department of Energy and the U.S. Food and Drug Administration. + +# Disclaimer +The mention of commercial products, their resources, or their use in connection with material reported herein is not to be construed as either an actual or implied endorsement of such products by the Department of Health and Human Services. This is a contribution of the U.S. Food and Drug Administration and not subject to copyright + +# References