[Feature Request] Synchronization between Dialogue Systems and Videos

[AdamSobieski]

I would like to share with the Microsoft Bot team and community a feature request and some ideas pertaining to synchronization between dialogue systems and video player components.

With synchronization between dialogue systems and videos, chatbots can be made to seem aware of the contents and contexts of related multimedia resources.

Scenarios for these features include education, e.g., instructional videos with accompanying chatbot support, and business, e.g., videos showcasing products with chatbots available to discuss them.

Synchronization between dialogue systems and videos can be bidirectional:

• dialogue systems could launch videos, switch videos, and control video playback, pausing, playing, and seeking to points in videos
• videos could send playback events, including text track cue events, to dialogue systems

Related topics include synchronization of HTML document events, e.g., scrolling-based events, with dialogue systems so that chatbots can ascertain dialogue contexts from users' positions in related documents. Similarly, chatbots could scroll to and highlight content in documents and receive events as document content comes into view.

Thank you. I hope that this feature request and related ideas are of some interest. I would like to request support for these scenarios, in particular synchronization capabilities between Web-based dialogue systems and related HTML documents and video player components.