Merge pull request #30 from beeldengeluid/remove-audio-from-similarity

removed audio features

_labo-help/similarity.markdown

@@ -14,9 +14,9 @@ This is the "Similarity" tool. In this brief help page you will:
 
 ## <a name="intro"></a>Intro
 
-The Similarity tool allows you to discover related keyframes from other media resources. Like in the resource viewer, you can watch a media resource, explore and navigate the content using visual and audio keyframes. In addition, related keyframes are retrieved and shown for every keyframe in the resource. 
+The Similarity tool allows you to discover related keyframes from other media resources. Like in the resource viewer, you can watch a media resource, explore and navigate the content using visual keyframes. In addition, related keyframes are retrieved and shown for every keyframe in the resource. 
 
- The tool enables researchers to browse a part of the archive in a way that is complementary to the structured search offered in the Search tool. This may expose relationships that are not apparent from the metadata, emerging from the visual and audio domain. These relations are often more associative (like a resemblance in composition or colours) than semantic (like picturing the same object or person). 
+The tool enables researchers to browse a part of the archive in a way that is complementary to the structured search offered in the Search tool. This may expose relationships that are not apparent from the metadata, emerging from the visual domain. These relations are often more associative (like a resemblance in composition or colours) than semantic (like picturing the same object or person). 
 
 Note that the current version of the tool is merely a proof of concept, based on a limited amount of [data](#data). It is being developed further in 2024, and a bigger part of the collection will be covered. 
 
@@ -30,17 +30,17 @@ The Similarity tool offers the following features and functionality.
 
 ### <a name="keyframes"></a>Current resource keyframes
 
-Keyframes for the resource were selected using the [scene detect library](https://www.scenedetect.com/). They are accompanied by spectrograms of 1 second audio centered around the keyframe.
+Keyframes for the resource were selected using the [scene detect library](https://www.scenedetect.com/). 
 
-The keyframes are shown in a grid under the media viewer. They provide a visual and auditory overview of the current video. When the video is playing, the currently active keyframe is highlighted. Clicking on a tile moves the player video position to the keyframe timestamp. This way one can navigate the video, in addition to the left/right buttons that move playback to the previous or next keyframe.
+The keyframes are shown in a grid under the media viewer. They provide a visual overview of the current video. When the video is playing, the currently active keyframe is highlighted. Clicking on a tile moves the player video position to the keyframe timestamp. This way one can navigate the video, in addition to the left/right buttons that move playback to the previous or next keyframe.
 
 The keyframes grid is also available in the resource viewer. It provides the same control for the media playback as in the similarity tool.
 
 ### <a name="related-keyframes"></a>Related keyframes
 
 Related keyframes are loaded for the active keyframe. Results can be controlled using the options available in the taskbar above the related keyframes. Consult the info icon behind each option to learn more about its use. 
 
-Based on the active keyframe in the left panel, similar keyframes are retrieved by comparing the distance between visual and auditory feature vectors that have been created using the [VisXP model](#technology). Search scores can be displayed by using the score button:
+Based on the active keyframe in the left panel, similar keyframes are retrieved by comparing the distance between visual feature vectors that have been created using the [VisXP model](#technology). Search scores can be displayed by using the score button:
 
 ![Similarity tool score button](/uploads/similarity-tool-score.jpg)
 
@@ -58,7 +58,7 @@ In addition, both the current resource and the resources belonging to the relate
 
 ## <a name="data"></a>Data
 
-Currently, only a small selection of the NISV collection has been processed. It contains resources about specific events, for example 9/11 attacks, refugees debate in 2015, and the Dutch ice skating tour called elfstedentocht.
+Currently, only a small selection of the NISV collection has been processed. It contains resources about specific events, for example 9/11 attacks, refugees debate in 2015, and the Dutch ice skating tour called Elfstedentocht.
 
 More data will be added in the future.
 
@@ -72,8 +72,8 @@ The trained model allows us to embed all video snippets (shots, in our case) in
 The following steps were taken to apply the model to videos in the NISV archive:
 
 - First, videos were segmented into shots using [scenedetect](https://www.scenedetect.com/).
-For every shot, a representative keyframe (image) is extracted together with the corresponding audio spectogram (based on 1 second of audio). 
+For every shot, a representative keyframe (image) is extracted.
 
-- Next, the model (a convolutional neural network) was applied to extract audio and visual features. 
+- Next, the model (a convolutional neural network) was applied to extract visual features. 
 
 - For similarity search, the nearest items in feature space are retrieved. The vector search itself is handled by an Elastic Search backend.