Skip to content
This repository has been archived by the owner on Dec 4, 2023. It is now read-only.
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: VergilGao/docker-mdc
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 6.2.2-r1
Choose a base ref
...
head repository: VergilGao/docker-mdc
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref

Commits on May 30, 2022

  1. update readme #55

    VergilGao committed May 30, 2022
    Copy the full SHA
    e2ddb6a View commit details

Commits on Sep 3, 2022

  1. Copy the full SHA
    44c96b0 View commit details

Commits on Sep 4, 2022

  1. 6.3.1

    VergilGao authored Sep 4, 2022
    Copy the full SHA
    4e7837d View commit details

Commits on Sep 5, 2022

  1. Update README.md

    VergilGao authored Sep 5, 2022
    Copy the full SHA
    91ada1c View commit details

Commits on Oct 9, 2022

  1. 6.3.2

    VergilGao authored Oct 9, 2022
    Copy the full SHA
    abcb715 View commit details

Commits on Dec 1, 2022

  1. Update to Python 3.10

    VergilGao committed Dec 1, 2022
    Copy the full SHA
    4de18eb View commit details
  2. update dependence

    VergilGao committed Dec 1, 2022
    Copy the full SHA
    8965583 View commit details
  3. 6.4.1

    VergilGao committed Dec 1, 2022
    Copy the full SHA
    d93dce6 View commit details

Commits on Feb 3, 2023

  1. mdc arguments支持

    加入mdc arguments支持,比如-m 3
    frosty5689 authored and VergilGao committed Feb 3, 2023
    Copy the full SHA
    d3f6438 View commit details

Commits on Feb 5, 2023

  1. update dependence

    VergilGao authored Feb 5, 2023
    Copy the full SHA
    7ca839b View commit details

Commits on Feb 9, 2023

  1. update to 6.5.1

    VergilGao authored Feb 9, 2023
    Copy the full SHA
    3aae8ec View commit details

Commits on Feb 27, 2023

  1. Update README.md

    VergilGao authored Feb 27, 2023
    Copy the full SHA
    38c6369 View commit details

Commits on Mar 19, 2023

  1. Update to 6.5.2

    VergilGao authored Mar 19, 2023
    Copy the full SHA
    9323137 View commit details

Commits on Mar 31, 2023

  1. add arm64

    VergilGao authored Mar 31, 2023
    Copy the full SHA
    8a479db View commit details

Commits on May 23, 2023

  1. Copy the full SHA
    6eea13f View commit details
  2. 移除 arm64

    VergilGao committed May 23, 2023
    Copy the full SHA
    aab122f View commit details
  3. Update README.md

    VergilGao authored May 23, 2023
    Copy the full SHA
    3b89dc2 View commit details
  4. update readme

    VergilGao committed May 23, 2023
    Copy the full SHA
    e697e8d View commit details

Commits on Jul 7, 2023

  1. feat: Add support for dlib-bin build on arm64 (#83)

    * feat: Add support for dlib-bin build on arm64
    
    * fix: Not found dist dir
    
    * chore: optimize arg and desc
    
    * build: Add arm64 release build
    
    * add arm64 image test
    
    * update docker action to v2 in release.yml
    
    ---------
    
    Co-authored-by: 羽先生 <8655163+VergilGao@users.noreply.github.com>
    NavyD and VergilGao authored Jul 7, 2023
    Copy the full SHA
    69c9533 View commit details
  2. build and push test images

    VergilGao committed Jul 7, 2023
    Copy the full SHA
    e3c7683 View commit details
  3. update mdc version

    VergilGao committed Jul 7, 2023
    Copy the full SHA
    637f158 View commit details

Commits on Jul 18, 2023

  1. Update Dockerfile

    VergilGao authored Jul 18, 2023
    Copy the full SHA
    6f7c5b8 View commit details
Showing with 146 additions and 183 deletions.
  1. +0 −39 .github/workflows/build-baseimage.yml
  2. +3 −3 .github/workflows/release.yml
  3. +31 −36 .github/workflows/test.yml
  4. +97 −26 Dockerfile
  5. +13 −5 README.md
  6. +0 −49 build-stage/Dockerfile
  7. +2 −2 docker-entrypoint.sh
  8. +0 −13 root/ldd
  9. +0 −10 root/pyinstaller.sh
39 changes: 0 additions & 39 deletions .github/workflows/build-baseimage.yml

This file was deleted.

6 changes: 3 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -21,10 +21,10 @@ jobs:
echo "BUILD_DATE=$(date +"%Y-%m-%d")" >> $GITHUB_ENV
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v2

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2

- name: Login to DockerHub
uses: docker/login-action@v1
@@ -45,7 +45,7 @@ jobs:
with:
context: .
file: ./Dockerfile
platforms: linux/amd64
platforms: linux/amd64,linux/arm64
push: true
tags: |
vergilgao/mdc:${{ env.MAJOR_VERSION }}
67 changes: 31 additions & 36 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -26,42 +26,37 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Create Test User and test files
- name: Set up build args
run: |
sudo groupadd -g 1024 mdc
sudo useradd -u 1024 mdc -g mdc
mkdir test
dd if=/dev/zero of="./test/MIFD-046.mp4" bs=250MB count=1
dd if=/dev/zero of="./test/SVDVD-471.mp4.mp4" bs=250MB count=1
sudo chown -R 99:100 test
ls -al test
echo "TAG_VERSION=${GITHUB_REF_NAME}" >> $GITHUB_ENV
echo "BUILD_DATE=$(date +"%Y-%m-%d")" >> $GITHUB_ENV
- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Build the test image
run: |
docker build -t vergilgao/mdc:test \
--build-arg BUILD_DATE=$(date +"%Y-%m-%d") \
.
- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: vergilgao
password: ${{ secrets.GHCR_TOKEN }}

- name: Run the Test
if: success()
run: |
docker run \
--rm \
--name=mdc_test \
-e UID=1024 \
-e GID=1024 \
-v ${PWD}/config:/config \
-v ${PWD}/test:/data \
vergilgao/mdc:test || true
docker run \
--rm \
--name=mdc_test \
-e UID=1024 \
-e GID=1024 \
-v ${PWD}/config:/config \
-v ${PWD}/test:/data \
vergilgao/mdc:test
find . -name MIFD-046.mp4
find . -name SVDVD-471.mp4
ls -al test
- name: Build and push
id: docker_build
uses: docker/build-push-action@v2
with:
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: |
ghcr.io/vergilgao/mdc:nightly-${{ github.sha }}
build-args: |
VERSION=nightly-${{ github.sha }}
BUILD_DATE=${{ env.BUILD_DATE }}
123 changes: 97 additions & 26 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,26 +1,96 @@
FROM ghcr.io/vergilgao/mdc-buildimage:dev as build-stage
ARG DLIB_DIR=/tmp/dlib
ARG DLIB_WHL_DIR=${DLIB_DIR}/dist

FROM python:3.10-slim-bullseye AS dlib-bin-builder-amd64
ARG DLIB_WHL_DIR
RUN mkdir -p ${DLIB_WHL_DIR}

FROM python:3.10-slim-bullseye AS dlib-bin-builder-arm64
# https://github.com/ageitgey/face_recognition/blob/master/Dockerfile
RUN apt-get -y update
RUN apt-get install -y --fix-missing \
build-essential \
cmake \
gfortran \
git \
wget \
curl \
graphicsmagick \
libgraphicsmagick1-dev \
libatlas-base-dev \
libavcodec-dev \
libavformat-dev \
libgtk2.0-dev \
libjpeg-dev \
liblapack-dev \
libswscale-dev \
pkg-config \
python3-dev \
python3-numpy \
software-properties-common \
zip \
&& apt-get clean && rm -rf /tmp/* /var/tmp/*

ARG DLIB_VERSION=v19.24.2
ARG DLIB_DIR

# dlib-bin repotory dlib-wheels steps: https://github.com/alesanfra/dlib-wheels/blob/master/.github/workflows/build.yaml
RUN mkdir -p ${DLIB_DIR} && \
git clone -b "${DLIB_VERSION}" --single-branch https://github.com/davisking/dlib.git ${DLIB_DIR} && \
cd ${DLIB_DIR} && \
# change dlib python module desc
sed -i'' -e "s/name='dlib'/name='dlib-bin'/" setup.py && \
sed -i'' -e "s/version=read_version_from_cmakelists('dlib\/CMakeLists.txt')/version='$DLIB_VERSION'/" setup.py && \
sed -i'' -e "s/url='https:\/\/github\.com\/davisking\/dlib'/url='https:\/\/github\.com\/navyd\/docker-mdc'/" setup.py && \
sed -i'' -e "s/_cmake_extra_options = \[\]/_cmake_extra_options = \['-DDLIB_NO_GUI_SUPPORT=ON'\]/" setup.py && \
# build dlib: https://github.com/davisking/dlib#compiling-dlib-python-api
pip install build && \
python -m build --wheel && \
# check
pip install ./dist/*.whl

FROM dlib-bin-builder-${TARGETARCH} AS dlib-bin-builder

FROM python:3.10-slim-bullseye as build-stage

RUN \
apt-get -y update && apt-get -y upgrade \
&& apt install -y -q \
bash \
wget \
binutils \
upx \
&& apt-get autoremove --purge -y \
&& apt-get clean -y

ARG MDC_SOURCE_VERSION
ENV MDC_SOURCE_VERSION=${MDC_SOURCE_VERSION:-a6aaca984f4f52e187d0f4c68424bb18a4895cb2}
ENV MDC_SOURCE_VERSION=${MDC_SOURCE_VERSION:-c3e5fdb09fefa11c614e519786e942b08b2a8fb0}

RUN mkdir -p /tmp/mdc && cd /tmp/mdc && \
ARG DLIB_WHL_DIR
COPY --from=dlib-bin-builder $DLIB_WHL_DIR $DLIB_WHL_DIR

RUN mkdir -p /tmp/mdc && cd /tmp/mdc \
# get mdc source code
wget -O- https://github.com/yoshiko2/Movie_Data_Capture/archive/$MDC_SOURCE_VERSION.tar.gz | tar xz -C /tmp/mdc --strip-components 1 && \
# fix dowload error
sed -i "s/if configProxy:/if configProxy.enable:/g" core.py && \
# build mdc
/pyinstaller/pyinstaller.sh \
--onefile Movie_Data_Capture.py \
&& wget -O- https://github.com/yoshiko2/Movie_Data_Capture/archive/$MDC_SOURCE_VERSION.tar.gz | tar xz -C /tmp/mdc --strip-components 1 \
&& python3 -m venv /opt/venv && . /opt/venv/bin/activate \
&& pip install --upgrade \
pip \
pyinstaller \
&& if [ -n "$(ls -A $DLIB_WHL_DIR)" ]; then pip install $DLIB_WHL_DIR/*.whl; fi \
&& pip install -r requirements.txt \
&& pip install face_recognition --no-deps \
&& pyinstaller \
-D Movie_Data_Capture.py \
--python-option u \
--hidden-import "ImageProcessing.cnn" \
--add-data "Img:Img" \
--add-data "$(python -c 'import cloudscraper as _; print(_.__path__[0])' | tail -n 1):cloudscraper" \
--add-data "$(python -c 'import opencc as _; print(_.__path__[0])' | tail -n 1):opencc" \
--add-data "$(python -c 'import face_recognition_models as _; print(_.__path__[0])' | tail -n 1):face_recognition_models"

FROM ghcr.io/vergilgao/alpine-baseimage
--add-data "$(python -c 'import face_recognition_models as _; print(_.__path__[0])' | tail -n 1):face_recognition_models" \
--add-data "Img:Img" \
--add-data "scrapinglib:scrapinglib" \
&& cp /tmp/mdc/config.ini /tmp/mdc/dist/Movie_Data_Capture/config.template

RUN apk --update --no-cache add \
libxcb
FROM debian:11-slim

ARG BUILD_DATE
ARG VERSION
@@ -36,18 +106,19 @@ ENV GID=100
ENV UMASK=002

ADD docker-entrypoint.sh docker-entrypoint.sh

RUN chmod +x docker-entrypoint.sh && \
mkdir -p /app && \
mkdir -p /data && \
mkdir -p /config && \
useradd -d /config -s /bin/sh mdc && \
chown -R mdc /config && \
chown -R mdc /data

COPY --from=build-stage /tmp/mdc/dist/Movie_Data_Capture /app
COPY --from=build-stage /tmp/mdc/config.ini /app/config.template

RUN \
apt-get -y update && apt-get -y upgrade \
&& apt install -y -q \
gosu \
&& apt-get autoremove --purge -y \
&& apt-get clean -y \
&& chmod +x docker-entrypoint.sh \
&& mkdir -p /data /config \
&& useradd -d /config -s /bin/sh mdc \
&& chown -R mdc /data /config

VOLUME [ "/data", "/config" ]

ENTRYPOINT ["/docker-entrypoint.sh"]
ENTRYPOINT ["/docker-entrypoint.sh"]
18 changes: 13 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Movie Data Capture - Docker

[github](https://github.com/VergilGao/docker-mdc) [![GitHub stars](https://img.shields.io/github/stars/VergilGao/docker-mdc)](https://github.com/VergilGao/docker-mdc/stargazers) | [dockerhub](https://hub.docker.com/repository/docker/vergilgao/mdc)
[github](https://github.com/VergilGao/docker-mdc) [![GitHub stars](https://img.shields.io/github/stars/VergilGao/docker-mdc)](https://github.com/VergilGao/docker-mdc/stargazers) | [dockerhub](https://hub.docker.com/r/vergilgao/mdc)

![publish](https://github.com/VergilGao/docker-mdc/workflows/publish/badge.svg) [![GitHub license](https://img.shields.io/github/license/VergilGao/docker-mdc)](https://github.com/VergilGao/docker-mdc/blob/master/LICENSE)
![release](https://github.com/VergilGao/docker-mdc/workflows/release/badge.svg) [![GitHub license](https://img.shields.io/github/license/VergilGao/docker-mdc)](https://github.com/VergilGao/docker-mdc/blob/master/LICENSE)

**数据无价,请谨慎操作!**

@@ -26,6 +26,14 @@

## 更新日志

20230523:

* 同步至上游 6.6.3
* 基础镜像修改为 debian11
* 移除 arm64 支持

因为上游人脸识别模块需要的依赖在 arm64 架构下需要编译,本人不擅长编译 pip 模块,所以放弃了 arm64 架构的维护。

20220529:

* 修复root用户运行
@@ -62,7 +70,7 @@
docker pull ghcr.io/vergilgao/mdc:latest
mkdir test
dd if=/dev/zero of="./test/MIFD-046.mp4" bs=250MB count=1
docker run --rm --name mdc_test -it -v ${PWD}/test:/data -v ${PWD}/config:/config -e UID=$(stat -c %u test) -e GID=$(stat -c %g test) vergilgao/mdc:latest
docker run --rm --name mdc_test -it -v ${PWD}/test:/data -v ${PWD}/config:/config -e UID=$(stat -c %u test) -e GID=$(stat -c %g test) ghcr.io/vergilgao/mdc:latest
```

然后你会看到如下输出:
@@ -132,8 +140,8 @@ docker run --rm -it \
--name mdc_test \
-v ${PWD}/test:/data \
-v ${PWD}/config:/config \
-e PUID=1000
-e PGID=1000
-e UID=1000 \
-e GID=1000 \
vergilgao/mdc:latest
```
49 changes: 0 additions & 49 deletions build-stage/Dockerfile

This file was deleted.

4 changes: 2 additions & 2 deletions docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -18,7 +18,7 @@ if [ ! -d /config ]; then
echo "---no config folder found, create...---"
mkdir -p /config
fi
chown -R ${UID}:${GID} /app /data /config
chown -R ${UID}:${GID} /data /config

echo "Checking if config file exist"
if [ ! -f "${config_file}" ]; then
@@ -30,4 +30,4 @@ fi

echo "Starting..."
cd /data
gosu ${USER} /app/Movie_Data_Capture
gosu ${USER} /app/Movie_Data_Capture
13 changes: 0 additions & 13 deletions root/ldd

This file was deleted.

10 changes: 0 additions & 10 deletions root/pyinstaller.sh

This file was deleted.