Dockerized Deployment V2

[Jamalarm]

Hey @vicwomg!

So I'm just taking a stab at this. I'm no expert in Python so I'm running into a few problems but I think the image is building mostly correctly. I made a few changes from the other PR:

• I decided to go with one of the official Python debian-based images (python:3.12-slim-bookworm) as most of your setup scripts and instructions assume you're on Debian
• I'm copying in the code directly from the repo, rather than pulling it from git. This is because eventually we should set up a Docker publish step as part of the Github release. Pulling the code from the local repo ensures that the published version matches and doesn't accidentally pull in code from a different release.
• I borrowed the same entrypoint.sh script from the other PR, but I changed it to call python3 /pikaraoke/pikaraoke/app.py directly rather than using the pikaraoke.sh script.

This is not currently working, as the app startup fails with:

Traceback (most recent call last):
2024-10-05T11:25:28.943391217Z   File "/pikaraoke/pikaraoke/app.py", line 36, in <module>
2024-10-05T11:25:28.943393384Z     from pikaraoke import VERSION, karaoke
2024-10-05T11:25:28.943394842Z ModuleNotFoundError: No module named 'pikaraoke'

Unfortunately this is where I run into the problem of not really knowing Python that well!

[vicwomg]

python3 /pikaraoke/pikaraoke/app.py is not the recommended way to launch pikaraoke from source locally. You can try installing poetry and running it that way. See: https://github.com/vicwomg/pikaraoke?tab=readme-ov-file#developing-pikaraoke

[Jamalarm]

Ok, adjusted it to use poetry and seems to be starting properly. The app now fails on any GET with:

Traceback (most recent call last):
2024-10-07T10:17:16.732223382Z   File "/root/.cache/pypoetry/virtualenvs/pikaraoke-dQFMxTlB-py3.12/lib/python3.12/site-packages/flask/app.py", line 2529, in wsgi_app
2024-10-07T10:17:16.732225340Z     response = self.full_dispatch_request()
2024-10-07T10:17:16.732226757Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-07T10:17:16.732228090Z   File "/root/.cache/pypoetry/virtualenvs/pikaraoke-dQFMxTlB-py3.12/lib/python3.12/site-packages/flask/app.py", line 1825, in full_dispatch_request
2024-10-07T10:17:16.732247090Z     rv = self.handle_user_exception(e)
2024-10-07T10:17:16.732249048Z          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-07T10:17:16.732250340Z   File "/root/.cache/pypoetry/virtualenvs/pikaraoke-dQFMxTlB-py3.12/lib/python3.12/site-packages/flask/app.py", line 1823, in full_dispatch_request
2024-10-07T10:17:16.732252048Z     rv = self.dispatch_request()
2024-10-07T10:17:16.732253298Z          ^^^^^^^^^^^^^^^^^^^^^^^
2024-10-07T10:17:16.732254548Z   File "/root/.cache/pypoetry/virtualenvs/pikaraoke-dQFMxTlB-py3.12/lib/python3.12/site-packages/flask/app.py", line 1799, in dispatch_request
2024-10-07T10:17:16.732256132Z     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
2024-10-07T10:17:16.732257507Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-07T10:17:16.732259090Z   File "/pikaraoke/pikaraoke/app.py", line 502, in splash
2024-10-07T10:17:16.732260632Z     status = subprocess.run(["iwconfig", "wlan0"], stdout=subprocess.PIPE).stdout.decode(
2024-10-07T10:17:16.732262507Z              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-07T10:17:16.732263965Z   File "/usr/local/lib/python3.12/subprocess.py", line 548, in run
2024-10-07T10:17:16.732265423Z     with Popen(*popenargs, **kwargs) as process:
2024-10-07T10:17:16.732266715Z          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-07T10:17:16.732267965Z   File "/usr/local/lib/python3.12/subprocess.py", line 1026, in __init__
2024-10-07T10:17:16.732269298Z     self._execute_child(args, executable, preexec_fn, close_fds,
2024-10-07T10:17:16.732270632Z   File "/usr/local/lib/python3.12/subprocess.py", line 1955, in _execute_child
2024-10-07T10:17:16.732272007Z     raise child_exception_type(errno_num, err_msg, err_filename)
2024-10-07T10:17:16.732273298Z FileNotFoundError: [Errno 2] No such file or directory: 'iwconfig'

iwconfig seems to be something for getting WLAN specific information about network adapters. I also found something about iwconfig being moved to /sbin in Buster onwards. I tried to fix this by adjusting the path but it doesn't seem to be working. Before I keep hitting my head against this, is iwconfig definitely needed in the image?

[vicwomg]

Not sure if I don't understand docker, but why don't you install pikaraoke via pip instead of pulling the source?
pip install pikaraoke

Then to run: pikaraoke

As for iwconfig, I don't know the exact reasoning behind it. Looks to have something to do with Wifi ap mode detection in raspberry pi devices only. Best to leave it in.

If your docker image doesn't have iwconfig, can't it simply be installed?

[Jamalarm]

why don't you install pikaraoke via pip instead of pulling the source

So if we eventually end up publishing the image to Docker Hub, you'll want to be publishing the specific version/code you're releasing on Github. If you look at something like MariaDB you can see all the version-specific tags like 11.1.6 and so on.

The way they do this is they'll have a CI pipeline that builds the source, packages it as an image, and then publishes it to Docker Hub all as part of the same pipeline. If you start pulling down from some remote package manager (like pip) at this point, you won't get the changes are currently releasing, so effectively the Docker build would be at least one version behind the whole time.

If your docker image doesn't have iwconfig, can't it simply be installed?

It does have it, but it looks like in later versions of Debian linux iwconfig is only available to sudo by default so it isn't on the regular PATH. I can probably get it working but was just checking if you happened to know if it was needed, though now as I look at it I realise that's actually being called from /usr/local/lib/python3.12/subprocess.py which I guess is some kind of core Python library and nothing to do with your code!

[vicwomg]

Got it that makes sense in regard to the versioning.

One could make the case that you could deploy to docker only after the version is released to pip, then have the docker config pull the proper version via pip install pikaraoke==X.X.X where X.X.X the newest version number. That would just be a CI pipeline configuration. But if this is more self-contained then great.

Regarding iwconfig It is being called by subprocess library, but this is indeed being done by pikaraoke. The underlying issue is iwconfig is either not in the PATH, or not installed. My suggestion would be to determine what directory it is installed to, and add that to the PATH variable in the docker config. If it's not installed, then have docker install it somehow where it can be accessed from the PATH.

[Jamalarm]

I've fixed the iwconfig problem. The interface now loads!

Video playing is not functioning. I thought at first it was because the 5556 port wasn't exposed, but it still isn't working after that.

It looks like something is assembling weird URLs (http://localhost:5555/://None:5556/1728380127) when specifying the -u localhost argument for local testing.

Omitting the -u argument results in what looks like a valid URL, but the host is the internal Docker hostname and isn't accessible outside of Docker:

[vicwomg]

-u / -url overrides the webserver url (splash screen and endpoints). There --ffmpeg-url parameter to override the stream url. Normally these are auto-detected, however in a containerized installation, as you noticed the IP address is invalid.

You'd need to ensure that both of these are a set to valid LAN IP address in order for other clients to connect. And that the port forwards correctly.

There was some discussion about this here. I think it involved some proxying of the two urls into a single endpoint. maybe you can dig up the solution.
#293

[Jamalarm]

Ok! Getting somewhere.

I can now access and display the /splash screen locally and on my smart TV's web browser over the network. My phone can successfully open the remote control app as well.

On my local web browser the video and sound plays perfectly, on my smart TV the interface displays properly and the sound plays for each video, but the video is black. I'm assuming this means the network is ok (otherwise we would get no sound) but probably some issue with the codec that is being streamed over to the browser?

I'm going to try tinkering with some settings to see if I can make it use a more basic codec format that may be supported by more embedded browsers.

Edit: adding --high-quality doesn't fix it.

[vicwomg]

Hmm, yeah embedded browsers might be a whole other thing. You should probably scope this to being able to get the docker container going.

…

On Tue, Oct 8, 2024 at 12:47 PM Tom ***@***.***> wrote: Ok! Getting somewhere. I can now access and display the /splash screen locally and on my smart TV's web browser over the network. My phone can successfully open the remote control app as well. On my local web browser the video and sound plays perfectly, on my smart TV the interface displays properly and the sound plays for each video, but the video is black. I'm assuming this means the network is ok (otherwise we would get no sound) but probably some issue with the codec that is being streamed over to the browser? I'm going to try tinkering with some settings to see if I can make it use a more basic codec format that may be supported by more embedded browsers. — Reply to this email directly, view it on GitHub <#403 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7KXNXCDLDB4MJUFINB5JDZ2QZGFAVCNFSM6AAAAABPNIRVTGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBQGY4TCNZYG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Jamalarm]

Yep, that makes sense.

Would you like to be the owner of the Docker Hub repo? I feel like that makes sense given you own the Github repo too. In which case could you do the setup steps here when you have a moment?

Once we have the repo we can add the Github Actions definition to build and publish the image on release.

[vicwomg]

Yup, I can own that.

In the meantime can you write up some basic instructions on how I can test this myself from your branch? Not too familiar with docker, but I do have it installed.

[Jamalarm]

Basic instructions to spin it all up locally:

1. Open a terminal at the repo root
2. Build and publish the docker container to your local repo with docker build --tag pikaraoke:latest (this takes a little while)
3. Modify docker-compose.yml, setting URL to either localhost (to test on the same machine) or whatever IP your dev machine has on your local network
4. In your terminal, run docker-compose up

Once the remote repo is up and running, only steps 3 and 4 will be required (and you won't need the repo cloned locally of course). Docker compose will automatically trigger an image pull from Docker Hub for whatever the latest version you published.

[vicwomg]

I had a look. Couldn't get it to work initially, but this command did it (add a .) : docker build . --tag pikaraoke:latest

I'm now able to run a headless server on localhost in docker. That seems to work fine.

However, is there a way to automatically pass the host LAN IP to the entrypoint command in the event of the absence of the URL? Ideally that would all be handled automatically and not be a manual configuration. Running headless server and client on localhost is not a common scenario.

[Jamalarm]

However, is there a way to automatically pass the host LAN IP to the entrypoint command in the event of the absence of the URL?
Yeah I was trying to figure that out. I think there's two options:

• Try and figure out some clever docker compose syntax that figures it out automatically and sets the environment variable
• Use a native docker networking feature. I think it might be possible with the macvlan network driver but I haven't used this before so need to do some experimentation.

Side point: I'm moving house over the next week so likely won't have time to make progress on this myself in that time. Will pick it up again when things calm down.

[vicwomg]

Thanks for the update, we're definitely close here.

I also saw this entry on a promising host.docker.internal reserved dns name that might achieve that. I tried plugging it in to entrypoint.sh and it didn't work though. Might not be accessible in sub scripts
https://docs.docker.com/desktop/networking/#i-want-to-connect-from-a-container-to-a-service-on-the-host

[vicwomg]

FWIW, I have a WIP branch where I'm trying to have pikaraoke run on a single port 5555, including ffmpeg streams. That would solve these issues.

I recall I got mostly there, but forget where I left off. Will revisit when I have the time.

[Jamalarm]

to have pikaraoke run on a single port 5555

Tbh I'm not sure this is really a problem. Docker is perfectly happy to expose as many ports as you like.

Apologies for not following up on this. I'm still in the chaos of setting up my new apartment so don't have any free time to play around with this stuff.