Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Update Docker and CI #2432

Closed
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .dive-ci
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
rules:
lowestEfficiency: 0.97 # ratio between 0-1
highestWastedBytes: 20MB # B, KB, MB, and GB
highestUserWastedPercent: 0.20 # ratio between 0-1
14 changes: 13 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
# ETC
.dive-ci
.editorconfig
.env
.hadolint.yaml
**/*.example
**/*.md
LICENSE

# Runtime
audio_cache/
bin/
@@ -8,13 +17,16 @@ musicbot/lib/__pycache__/

# Docker
.dockerignore
docker-compose.example.yml
docker-compose.yml
Dockerfile

# Git
.git/
.gitattributes
.github/
.gitignore

# IDE
.idea/
.vscode/
.vscode/
4 changes: 4 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
COMPOSE_FILE=docker-compose.yml
COMPOSE_REMOVE_ORPHANS=true
# * Options: linux/amd64 / linux/arm64/v8
PLATFORM=linux/arm64/v8
102 changes: 102 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
name: CI

on:
push:
branches:
- 'main'
- 'master'
tags:
- '*.*.*'
paths:
- 'Dockerfile*'
- 'pyproject.toml'
- 'poetry.lock'
- 'requirements.txt'
- '**.py'
- '**.sh'
- '.dockerignore'
- '.env.example'
- '.github/workflows/**'
Comment on lines +10 to +19
Copy link
Member

@srhinos srhinos Oct 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what will this actually do? this seems largely pointless and should just be removed

workflow_dispatch:

env:
REGISTRY_URL: ${{ vars.REGISTRY_URL }}
REGISTRY_USER: ${{ vars.REGISTRY_USER }}

jobs:
build:
name: Build and push Docker image
runs-on: ubuntu-latest
strategy:
fail-fast: true
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set password by container registry
run: |
case "${{ env.REGISTRY_URL }}" in
"ghcr.io")
echo "REGISTRY_PASS=${{ secrets.GITHUB_TOKEN }}" >> $GITHUB_ENV
;;
*)
if [ -n "${{ secrets.REGISTRY_PASS }}" ]; then
echo "REGISTRY_PASS=${{ secrets.REGISTRY_PASS }}" >> $GITHUB_ENV
else
echo "REGISTRY_PASS secret is not set and registry is not recognized. Exiting..."
exit 1
fi
;;
esac
- name: Log into container registry
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY_URL }}
username: ${{ env.REGISTRY_USER }}
password: ${{ env.REGISTRY_PASS }}

- name: Set image name
id: image_name
run: |
if [ -n "${{ env.IMAGE }}" ]; then
IMAGE="${{ env.IMAGE }}"
else
IMAGE=$(grep "LABEL org.opencontainers.image.title" Dockerfile | cut -d'"' -f2)
fi
echo "IMAGE=$IMAGE" >> $GITHUB_OUTPUT
echo "IMAGE=$IMAGE" >> $GITHUB_ENV
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY_URL }}/${{ env.REGISTRY_USER }}/${{ steps.image_name.outputs.IMAGE }}
tags: |
type=schedule
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
type=sha
type=raw,value=latest,enable={{is_default_branch}}
- name: Setup QEMU
uses: docker/setup-qemu-action@v3

- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64,linux/arm64/v8
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=${{ env.REGISTRY_URL }}/${{ env.REGISTRY_USER }}/${{ steps.image_name.outputs.IMAGE }}:buildcache
cache-to: type=registry,ref=${{ env.REGISTRY_URL }}/${{ env.REGISTRY_USER }}/${{ steps.image_name.outputs.IMAGE }}:buildcache,mode=max
Comment on lines +36 to +102
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is all so insanely complex for a simple task such as uploading a built docker image to github's container registry.

I built this exact workflow in another repo I maintain @ here, feel free to use it as inspiration as this workflow is really awful.

23 changes: 18 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,15 +1,24 @@
# editor settings
.idea/
.vscode/

# python bytecode
*.pyc

# temp files
~*/
.vscode/
*.service

# directories
audio_cache/
dectalk/

discord.log
logs/
data/
media/

# logs
discord.log

# configs
config/options.ini
config/permissions.ini
config/aliases.json
@@ -21,6 +30,10 @@ config/blacklist.txt
config/blocklist_users.txt
config/blocklist_songs.txt
config/playlists/
media/
config/autoplaylist.cachemap.json

# docker
docker-compose.yml

# inclusions (has to be declared last)
!**/*.example
12 changes: 12 additions & 0 deletions .hadolint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
failure-threshold: info # error|warning|info|style|ignore|none

ignored:
- DL3008 # pin versions in apt
- DL3013 # pin versions in pip
- DL3018 # pin versions in apk
- DL3042 # pip --no-cache-dir

trustedRegistries:
- docker.io
- "*.gcr.io"
- localhost:32000
39 changes: 27 additions & 12 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
FROM python:3.8-alpine
# syntax=docker/dockerfile:1.7.0

FROM python:3.8-alpine3.20
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any reason to deeply specify the alpine version?


# Add project source
WORKDIR /musicbot
@@ -7,22 +9,33 @@ COPY ./config sample_config

# Install build dependencies
RUN apk update && apk add --no-cache --virtual .build-deps \
build-base \
libffi-dev \
libsodium-dev
build-base \
libffi-dev \
libsodium-dev \
&& rm -rf /var/cache/apk/*

# Install dependencies
RUN apk update && apk add --no-cache \
ca-certificates \
ffmpeg \
opus-dev \
libffi \
libsodium \
gcc \
git
ca-certificates \
ffmpeg \
gcc \
git \
libffi \
libsodium \
opus-dev \
&& rm -rf /var/cache/apk/*

# pip env vars
ENV PIP_NO_CACHE_DIR=off
ENV PIP_DISABLE_PIP_VERSION_CHECK=on
ENV PIP_DEFAULT_TIMEOUT=100
Comment on lines +28 to +31
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these aren't needed as well. No reason to add them


# don't generate .pyc, enable tracebacks on seg faults
ENV PYTHONDONTWRITEBYTECODE=1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we dont want this enabled. It greatly reduces performance

ENV PYTHONFAULTHANDLER=1

# Install pip dependencies
RUN pip3 install --no-cache-dir -r requirements.txt
RUN python -m pip install --no-cache-dir -r requirements.txt

# Clean up build dependencies
RUN apk del .build-deps
@@ -33,3 +46,5 @@ VOLUME ["/musicbot/audio_cache", "/musicbot/config", "/musicbot/data", "/musicbo
ENV APP_ENV=docker

ENTRYPOINT ["/bin/sh", "docker-entrypoint.sh"]

LABEL org.opencontainers.image.title="musicbot"
44 changes: 0 additions & 44 deletions config/example_options.ini
Original file line number Diff line number Diff line change
@@ -12,15 +12,6 @@ Token = bot_token
Spotify_ClientID =
Spotify_ClientSecret =

# Sets the YouTube API Client ID, used by Yt-dlp OAuth2 plugin.
# Optional, unless built-in credentials are not working.
YtdlpOAuth2ClientID =

# Sets the YouTube API Client Secret key, used by Yt-dlp OAuth2 plugin.
# Optional, unless YtdlpOAuth2ClientID is set.
YtdlpOAuth2ClientSecret =


[Permissions]
# This option determines which user has full permissions and control of the bot.
# Only one user can be the bot's owner. You can generally leave this as "auto".
@@ -141,10 +132,6 @@ DeleteInvoking = no
# resume from where it left off.
PersistentQueue = yes

# Enable MusicBot to download the next song in the queue while a song is playing.
# Currently this option does not apply to auto-playlist or songs added to an empty queue.
PreDownloadNextSong = yes

# Determines what messages are logged to the console. The default level is INFO, which is
# everything an average user would need. Other levels include CRITICAL, ERROR, WARNING,
# DEBUG, VOICEDEBUG, FFMPEG, NOISY, and EVERYTHING. You should only change this if you
@@ -165,9 +152,6 @@ DebugLevel = INFO
# {p0_url} = The track url for the currently playing track.
StatusMessage =

# If enabled, status message updates will count and report paused players.
StatusIncludePaused = no

# Write what the bot is currently playing to the data/<server id>/current.txt FILE.
# This can then be used with OBS and anything else that takes a dynamic input.
WriteCurrentSong = no
@@ -268,34 +252,6 @@ SavePlayedHistoryGuilds = no
# to play files from the local MediaFileDirectory path.
EnableLocalMedia = no

# Allow MusicBot to automatically unpause when play commands are used.
UnpausePlayerOnPlay = no

# Experimental, HTTP/HTTPS proxy settings to use with ytdlp media downloader.
# The value set here is passed to `ytdlp --proxy` and aiohttp header checking.
# Leave blank to disable.
YtdlpProxy =

# Experimental option to set a static User-Agent header in yt-dlp.
# It is not typically recommended by yt-dlp to change the UA string.
# For examples of what you might put here, check the following two links:
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent
# https://www.useragents.me/
# Leave blank to use default, dynamically generated UA strings.
YtdlpUserAgent =

# Experimental option to enable yt-dlp to use a YouTube account via OAuth2.
# When enabled, you must use the generated URL and code to authorize an account.
# The authorization token is then stored in the `data/auth.token` file.
# This option should not be used when cookies are enabled.
# Using a personal account may not be recommended.
YtdlpUseOAuth2 = no

# Optional youtube URL used at start-up for triggering OAuth2 authorization.
# This starts the OAuth2 prompt early, rather than waiting for a song request.
# Authorization must be completed before start-up will continue when this is set.
YtdlpOAuth2URL =


[Files]
# Configure automatic log file rotation at restart, and limit the number of files kept.
16 changes: 6 additions & 10 deletions config/example_permissions.ini
Original file line number Diff line number Diff line change
@@ -73,18 +73,14 @@
; that the bot is already joined in the server. It is also expected that the user have ability to invoke summon command to
; use this option.
;
; Extractors = spotify:musicbot youtube generic soundcloud Bandcamp
; Extractors = spotify:musicbot youtube youtube:playlist youtube:tab youtube:search
; Specify yt-dlp extractor names that MusicBot will allow users to play media from.
; Each extractor name should be separated by spaces or commas.
; If left empty, hard-coded defaults will be allowed.
; The yt-dlp project has a list of supported services / extractor names here:
; Each extractor name should be separated by spaces.
; If left empty, all services will be allowed. Including porn services.
; The yt-dlp project has a list of supported services here:
; https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md
;
; The extractor `spotify:musicbot` is provided by MusicBot, not by yt-dlp.
; To allow ALL services, including porn services, add "__" to the list, without quotes.
; Example to allow all:
;
; Extractors = __
; The extractor `spotify:musicbot` is provided by MusicBot, not yt-dlp.
;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

@@ -134,7 +130,7 @@ SkipWhenAbsent = no
BypassKaraokeMode = no
SummonNoVoice = no
SkipLooped = no
Extractors = generic youtube spotify:musicbot Bandcamp soundcloud
Extractors = generic youtube youtube:playlist youtube:tab youtube:search spotify:musicbot

; This group has full permissions.
[MusicMaster]
5 changes: 1 addition & 4 deletions config/i18n/en.json
Original file line number Diff line number Diff line change
@@ -14,15 +14,12 @@
"cmd-save-exists": "This song is already in the autoplaylist.",
"cmd-save-invalid": "There is no valid song playing.",
"cmd-save-success": "Added <{0}> to the autoplaylist.",
"cmd-save-success-multiple": "Added {0} songs to the autoplaylist.",
"cmd-unsave-does-not-exist": "This song is not yet in the autoplaylist.",
"cmd-unsave-success": "Removed <{0}> from the autoplaylist.",
"cmd-autoplaylist-does-not-exist": "This song is not yet in the autoplaylist.",
"cmd-autoplaylist-invalid": "The supplied song link is invalid.",
"cmd-autoplaylist-option-invalid": "Invalid option \"{0}\" specified, use +, -, add, or remove",
"cmd-autoplaylist-success": "Removed <{0}> from the autoplaylist.",
"cmd-autoplaylist-add-all-empty-queue": "The queue is empty. Add some songs with `{0}play`!",
"cmd-save-all-exist": "All songs in the queue are already in the autoplaylist.",
"cmd-joinserver-response": "Click here to add me to a server: \n{}",
"cmd-play-spotify-album-process": "Processing album `{0}` (`{1}`)",
"cmd-play-spotify-album-queued": "Enqueued `{0}` with **{1}** songs.",
@@ -84,7 +81,7 @@
"cmd-resume-reply": "Resumed music in `{0.name}`",
"cmd-resume-none": "Player is not paused.",
"cmd-shuffle-reply": "Shuffled `{0}`'s queue.",
"cmd-clear-reply": "Cleared `{0}'s` queue",
"cmd-clear-reply": "Cleared `{0}`'s queue",
"cmd-remove-none": "There's nothing to remove!",
"cmd-remove-reply": "Removed `{0}` added by `{1}`",
"cmd-remove-missing": "Nothing found in the queue from user `%s`",
Loading