Possibility to link local videos/audio to grid element

[klues]

Currently it's not possible to include videos or audio from the local device to a grid element. While short audio could be implemented to be saved within the local/online database (see #22 ), longer audio (e.g. music) or videos probably cannot be saved within the database, because the're too big and we don't have the server ressources for that.

However technically there is the possibility to also access the local file system using the File System Access API. The drawback here is that currently it isn't possible to give an PWA (like AsTeRICS Grid is) persistent access to the file system, but the browser will always ask again if it's OK to access the files, after the app has been restarted.
However at some point maybe it will be possible to give persistent access to a folder for installed PWAs, see WICG/file-system-access#238 (comment)

Currently it would be possible to implement it like this:

• The user can select a directory where AG has access and where he can put into videos/audio he wants to link to AG (if he wants and trusts AG, also the complete HDD so that there is no need to copy files to a different folder).
• Media within this directory can be linked to a grid element
• Drawback: every time AG is started, the user has to confirm that AG is still allowed to access the folder.
• Limitation: File System Access API is currently only supported by desktop Chromium based browsers, see https://caniuse.com/native-filesystem-api

See this example text editor open a text file and save it, so see what the prompt looks like.
Given the limitation for desktop Chromium browsers (which I just found out while writing) I think this solution currently isn't really applicable. So maybe we'll have to wait until browser support will be better or find other possibilities. E.g. it would be possible to create a second local database which isn't synced and contains the big files. However this again would have it's own problems and limitations. Another idea would be a bridge application which handles files from the local file system to AG.

[ChrisVeigl]

hmmm - this is indeed a severe limitiation of the WebApp ...
adding local audio/video content would be an important improvement!

I just tried the demo app text editor and it worked fine without confirmation in Chrome and - strange enough - also in Firefox!

Probably it won't work on Android or iOS though... if this is so I think we should investige the "local bridge app" approach!

[deinhofer]

While the File System Access API is probably the best choice for long term, I think there are some good workarounds for now:
As you mentioned a bridging application could do the job. Here are some ideas:

Asterics + ApplicationLauncher or MediaPlayer plugin

This way you can directly start/stop the playback of a file with a local media player application.

Cons

Many steps involved:

1. install Asterics
2. install and configure media player
3. configure media library
4. need to know file paths when assigning media file to grid cell

VLC with http interface

VLC has an optional http interface which makes it remote controllable via http.
On my computer I tried:

vlc -I http --http-password test --media-library /media/mad/Daten/Users/mad/Videos/

Then I could access the web-based media player using http://localhost:8080
For better UI integration into AG it's probably better to directly access the http URL of the media file.

Pros

You can define a play list and trigger playback remotely.

Cons

1. Need to install vlc
2. need to know file paths when assigning media file to grid cell

Security issues

vlc allows setting an http password, which is probably not very secure but helps a bit. If the media directory is set explicitly exposing unintended files should not happen.

Asterics web server

Asterics provides a built-in web-server which could be used to expose media files via http or https.
You will simply have to copy your media files to the document root of the installed asterics application e.g.
C:\AsTeRICS\web\music

You can then access the single media files using http://localhost:8081/music/my-song.mp3 or https://localhost:8083/music/my-song.mp3

Cons

1. Need to install asterics
2. need to copy media files to document root
3. need to know file paths when assigning media file to grid cell

Security issues

While the asterics web server does not have a password the CORS settings should only allow access from *.asterics.eu domains. Anyway only the files of the document root are exposed.

Other protocols

Other options could be to install a local ftp, WebDAV or media (streaming) server.

[klues]

I just tried the demo app text editor and it worked fine without confirmation in Chrome and - strange enough - also in Firefox!

In Firefox they aren't using the File System Access API, see the "Lecacy Mode" note at the bottom right. You only see the confirmation prompt if you're trying to save a changed file or re-open a recent one (Tab "Recent"). The second thing would be needed by AsTeRICS Grid - re-open a file/folder that once was selected earlier.

Thanks @deinhofer for the detailed list of possible alternatives. I think "AsTeRICS + Application Launcher" is currently already possible using "ARE actions". But I think for the majority of people all proposed solutions are too complex. An additional idea for me is to add a general purpose "HTTP request" action, where AG can send custom HTTP requests at selecting a cell. This would also make the solution with VLC possible. However I don't think I would add a special "Play media" action that is then that complicated to configure. I think for that I would wait for the File System Access API to get better support.

As an alternative for the File System Access API, maybe from UX perspective the best approach would be to store big media files in a separate IndexedDB database, which isn't synced. The only limitation users will notice is that the files aren't accessible on other devices (which would also be the case in all other scenarios), but no additional configuration from user side would be needed. The technical drawback is that all the files will be saved in duplicated form on the user's harddisk.

[deinhofer]

I agree that a general purpose HTTP request would be sufficient to access a media file. This way several scenarios are possible

• hosting files locally with vlc or asterics:
  • pros: works offline
  • cons: probably does not work on Android and iOS
• hosting files on dropbox, google drive, OneDrive or Youtube and providing private links
  • pros: works on all platforms
  • cons: only works online, privacy

For any scenario you will have to deploy the media files before and copy paste a link to the cell configuration.
I don't see the big advantage for an extra IndexedDB here as it does not support synchronization and you would have to provide features to organize your media files within AG. Or how would the files get there? By drag and drop or import?

[klues]

I think we have a different understanding for "general purpose HTTP request". What I was talking about is the ability to trigger custom HTTP requests (e.g. in order to control VLC), and I think what you meant in your comment is the possibility to fetch and play custom media files that are served via HTTP.

I don't see the big advantage for an extra IndexedDB here as it does not support synchronization and you would have to provide features to organize your media files within AG.

The big difference would be that it works locally and without any additional user-side configuration. People could select files from their local file system and link them to a grid element (while it's internally copied to the IndexedDB).

[ChrisVeigl]

as many users work on Android or iOS, relying on an AsTeRICS installation is IMO not an option!

I was thinking about a small native bridge-application which just forwards the local media content to AG. It would be necesssary to develop this "add-on" for every OS, and users would need to install it seperately if they want the extra functions. An advantage could be that also other native device functions could be provided by this bridge.

[klues]

Yes, I was already thinking of (and starting a prototype) for such a native bridge for features like sending text messages via Signal. This also could be used for media access. However until we don't have more people in the development team of AsTeRICS Grid, it's not realistic to implement something like this for all plattforms.

But with File Access API the day will come when a bridge isn't needed any more, so that's the second reason I don't think it makes much sense to put too much effort in this bridge. And if not, the local IndexedDB solution is still much easier to implement and also better UX for the users since they don't need any additional software.

[ChrisVeigl]

makes sense!
i agree that using a local indexedDB is the best solution by now!