Develop merge V3

Dockerfile

@@ -1,36 +1,45 @@
-# Use an official Python runtime as a parent image
-FROM python:3
+FROM python:3.12.4-alpine3.20
+
+# Install system dependencies
+RUN apk add --no-cache \
+    bash \
+    curl \
+    git \
+    dcron \
+    ffmpeg \
+    tzdata \
+ && addgroup -S appgroup \
+ && adduser -S appuser -G appgroup
+
+# Set the timezone
+ENV TZ=Europe/Kiev
+RUN ln -sf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
 
-# Set the working directory in the container
 WORKDIR /app
 
+# Install Python dependencies
 COPY requirements.txt ./
 RUN pip install --no-cache-dir -r requirements.txt
 
-# Copy the Python script and any other necessary files
 COPY . .
 
-# Add the config folder as a volume
+# Define volumes
 VOLUME /app/data
+VOLUME /app/downloads
 
-# Define the default cron schedule
-ENV CRON_SCHEDULE="0 8 * * *"
+# Set file ownership
+RUN chown -R appuser:appgroup /app
 
-# Add the cron job to run toloka2transmission
-ADD crontab /etc/cron.d/cron-job
-RUN chmod 0644 /etc/cron.d/cron-job
-RUN touch /var/log/cron.log
+# Setup environment
+ENV PORT=5000 \
+    CRON_SCHEDULE="0 8 * * *"
 
-# Start cron service
-CMD cron && tail -f /var/log/cron.log
-
-# Make port available to the world outside this container
-EXPOSE 5000
-
-# Define environment variable
-ENV PORT 5000
-
-# Start-up script to run both cron and the web server
+# Prepare the entrypoint
 COPY docker-entrypoint.sh /usr/local/bin/
 RUN chmod +x /usr/local/bin/docker-entrypoint.sh
-ENTRYPOINT ["docker-entrypoint.sh"]
+
+# Switch to the non-root user
+USER appuser
+
+ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
+CMD ["crond", "-f", "-d", "8"]

README.md

@@ -25,13 +25,13 @@ Here is the roadmap for the project, outlining both completed and planned develo
 - [x] **Docker** - Simplify deployment and updates.
 - [x] **Search, Add, Download Results from Toloka** - Implemented functionality to search, add, and download content directly from Toloka.
 - [x] **Add Assistant to Adjust Naming** - Developed an assistant to help with file naming conventions, ensuring consistency across downloads.
-- [x] **Studios and Anime Local DB Preview** - Created a local database for studios and anime to allow quick previews and access.
-- [ ] **Jobs Schedule** - Plan to implement a scheduling system for automated tasks and jobs within the application.
+- [x] **Studios and Anime Local DB Preview** - DB based on streaming sites auto-downloaded on first application start
+- [x] **Jobs Schedule** - Cron job inside docker config.
 - [ ] **Integration with Stream2MediaServer to Replicate Logic for Online Cinemas** - Future plans to integrate with Stream2MediaServer to handle content from online cinemas similarly.
 - [ ] **Improved Parsing and Result Linking** - Aiming to enhance the parsing algorithms and result linking for better accuracy and user experience.
 - [ ] **Migration to Single DB** - Plans to consolidate multiple databases into a single, more efficient database system.
-- [ ] **MAL, TVDB, TheMovieDB Integration to Fetch Info** - Intend to integrate with external APIs like MAL (MyAnimeTable), TVDB (The TV Database), and TheMovieDB to fetch and display additional information.
-- [ ] **UI Improvements** - Scheduled improvements on the user interface to enhance usability and aesthetics.
+- [x] **MAL, TheMovieDB Integration to Fetch Info** - MAL and TheMovieDB results in multi-search
+- [ ] **UI Improvements** - improvements on the user interface to enhance usability and aesthetics.
 
 
 ## Розгортання за допомогою Docker
@@ -46,56 +46,31 @@ tags:
 
 copy configs and db into /path/to/config/
 
-### Використання Docker
+### Використання Docker(Portainer)
 
-1. **Завантаження образу Docker**
-   Завантажте готовий образ з Docker Hub за допомогою наступної команди:
-
-   ```bash
-   docker pull maksii/toloka2web:latest
-   ```
-
-
-2. **Запуск контейнера**
-   Використовуйте наступну команду для запуску контейнера:
-
-   ```bash
-   docker run -d -p 5000:5000 -v /path/to/your/config:/app/data --name toloka maksii/toloka2web:latest
-   ```
-
-   Замініть `/path/to/your/config` на шлях до вашої папки конфігурації.
-
-### Використання Portainer
-
-Якщо ви використовуєте Portainer для управління контейнерами Docker, ви можете легко розгорнути `Toloka2Web` як стек:
-
-1. **Логін в Portainer**
-   Увійдіть у вашу панель керування Portainer .
-
-2. **Створення стека**
-   Перейдіть до розділу "Stacks" і натисніть "Add Stack".
-
-3. **Конфігурація стека**
-   Дайте ім'я вашому стеку і вставте наступний YAML конфіг у поле "Web editor":
+**Конфігурація стека**
 
    ```yaml
-   version: '3.8'
-   services:
-     Toloka2Web:
-       image: maksii/toloka2web:latest
-       ports:
-         - "5000:5000"
-       volumes:
-         - /path/to/your/config:/app/data
-       restart: unless-stopped
+version: '3.8'
+services:
+  toloka2web:
+    image: maksii/toloka2web:latest
+    container_name: toloka2web
+    volumes:
+      - /path/to/your/config:/app/data
+       - /path/to/your/downloads:/path/to/your/downloads
+    environment:
+      - PORT=80 
+      - PUID=1024
+      - PGID=100
+      - CRON_SCHEDULE=0 */2 * * *
+      - TZ=Europe/Kiev
+    restart: unless-stopped
+    user: "${PUID}:${PGID}"
    ```
 
    Замініть `/path/to/your/config` на шлях до вашої папки конфігурації.
 
-4. **Розгортання стека**
-   Натисніть "Deploy the stack" для запуску вашого додатку.
-
-
 ## Disclaimer
 
 This project is designed with a simplified technology stack to maximize accessibility and encourage contributions from a wide range of developers, regardless of their experience level. The current technology choices, including the use of Flask and Python, SQLite for database, and vanila JavaScript for frontend interactions, have been deliberately chosen to keep the project straightforward and manageable.

app/routes/mal.py

@@ -1,13 +1,16 @@
 from flask import Blueprint, request, jsonify
+from flask_login import login_required
 from app.services.mal_service import search_anime, get_anime_detail
 
 mal_bp = Blueprint('mal_bp', __name__)
 
 @mal_bp.route('/api/mal/search', methods=['GET'])
+@login_required
 def search():
     query = request.args.get('query')
     return jsonify(search_anime(query))
 
 @mal_bp.route('/api/mal/detail/<int:anime_id>', methods=['GET'])
+@login_required
 def get_detail(anime_id):
     return jsonify(get_anime_detail(anime_id))

app/routes/release.py

@@ -1,6 +1,7 @@
 from flask import Blueprint, jsonify, request, make_response
 from flask_login import login_required
 
+from app.services.route_service import login_or_api_key_required
 from app.services.services import add_release_logic, get_titles_logic, update_all_releases_logic, update_release_logic
 
 release_bp = Blueprint('release', __name__)
@@ -29,6 +30,6 @@ def update_release():
     return jsonify(update_release_logic(request.form))
 
 @release_bp.route('/api/releases/', methods=['POST'])
-@login_required
+@login_or_api_key_required 
 def update_all_releases():
     return jsonify(update_all_releases_logic())

app/routes/routes.py

@@ -11,6 +11,7 @@
 from app.app import db 
 
 from ..services.services import (
+    multi_search,
     proxy_image_logic
 )
 
@@ -29,6 +30,12 @@ def settings():
     def proxy_image():
         url = request.args.get('url')
         return proxy_image_logic(url)
+
+    @app.route('/api/search')
+    @login_required
+    def search_aggregated():
+        query = request.args.get('query')
+        return multi_search(query)
 
     @login_manager.user_loader
     def load_user(user_id):
@@ -93,4 +100,11 @@ def on_identity_loaded(sender, identity):
 
         # Assuming the User model has a list of roles, update the identity with the roles that the user provides
         if hasattr(current_user, 'roles'):
-            identity.provides.add(RoleNeed(current_user.roles))
+            identity.provides.add(RoleNeed(current_user.roles))
+
+    #As user can use remember me, just a hacki fix to be sure, that hi identity updated before each request
+    #require some testing overtime to see if it will change behavior
+    @app.before_request
+    def before_request():
+        if current_user.is_authenticated:
+            identity_changed.send(current_app._get_current_object(), identity=Identity(current_user.id))

app/routes/settings.py

@@ -3,6 +3,7 @@
 from flask_principal import Permission, UserNeed, RoleNeed, identity_changed, Identity, AnonymousIdentity, identity_loaded
 
 from app.services.config_service import add_new_setting, read_all_settings_from_db, update_setting
+from app.services.route_service import get_installed_packages, list_files
 
 
 setting_bp = Blueprint('setting', __name__)
@@ -24,4 +25,17 @@ def add(setting_id):
 @login_required
 @admin_permission.require(http_exception=403)
 def update(setting_id):    
-    return jsonify(update_setting(setting_id, request.form['section'], request.form['key'], request.form['value'] ))
+    return jsonify(update_setting(setting_id, request.form['section'], request.form['key'], request.form['value'] ))
+
+@setting_bp.route('/api/settings/versions', methods=['GET'])
+@login_required
+@admin_permission.require(http_exception=403)
+def versions():
+    packages = get_installed_packages()
+    return jsonify(packages)
+
+@setting_bp.route('/api/settings/files', methods=['GET'])
+@login_required
+@admin_permission.require(http_exception=403)
+def check_config_files():
+    return list_files("app/data")

app/routes/stream.py

@@ -1,4 +1,5 @@
 from flask import Blueprint, jsonify, request
+import jsonpickle
 
 from app.services.services import add_title_from_streaming_site, search_titles_from_streaming_site
 
@@ -8,7 +9,7 @@
 @stream_bp.route('/api/stream', methods=['GET'])
 def search_titles_from_streaming():
     query = request.args.get('query')
-    return jsonify(search_titles_from_streaming_site(query))
+    return jsonpickle.encode(search_titles_from_streaming_site(query))
 
 @stream_bp.route('/api/stream', methods=['POST'])
 def add_title_from_streaming():

app/routes/tmdb.py

@@ -1,13 +1,22 @@
 from flask import Blueprint, request, jsonify
-from app.services.tmdb_service import search_movie, get_movie_detail
+from flask_login import login_required
+from app.services.tmdb_service import get_media_detail, get_trending_by_type, search_media
 
 tmdb_bp = Blueprint('tmdb_bp', __name__)
 
 @tmdb_bp.route('/api/tmdb/search', methods=['GET'])
+@login_required
 def search():
     query = request.args.get('query')
-    return jsonify(search_movie(query))
+    return jsonify(search_media(query))
 
-@tmdb_bp.route('/api/tmdb/detail/<int:movie_id>', methods=['GET'])
-def get_detail(movie_id):
-    return jsonify(get_movie_detail(movie_id))
+@tmdb_bp.route('/api/tmdb/detail/<int:id>', methods=['GET'])
+@login_required
+def get_detail(id):
+    query = request.args.get('type')
+    return jsonify(get_media_detail(id, query))
+
+@tmdb_bp.route('/api/tmdb/trending', methods=['GET'])
+def get_trending():
+    query = request.args.get('type')
+    return jsonify(get_trending_by_type(query))

app/routes/toloka.py

@@ -11,7 +11,13 @@
 def get_torrents():
     try:
         query = request.args.get('query')
-        return jsonify(get_torrents_logic(query))
+        response = get_torrents_logic(query)
+
+        # Check if the response is a "No results found" message
+        if isinstance(response, str) and response.startswith("No results found"):
+            return jsonify({"error": response})
+        else:
+            return jsonify(response)
     except Exception as e:
         # Return a custom JSON error message with a 500 Internal Server Error status
         error_message = {

app/services/mal_service.py

@@ -10,16 +10,16 @@ def search_anime(query):
     api_key = get_api_key('mal_api')
     if not api_key:
         return {'error': 'API key not found'}
-    url = f"https://api.myanimelist.net/v2/anime?q={query}"
-    headers = {'Authorization': f'Bearer {api_key}'}
+    url = f"https://api.myanimelist.net/v2/anime?q={query}&limit=10&fields=id,title,main_picture,alternative_titles,media_type,status,start_date,end_date"
+    headers = {'X-MAL-CLIENT-ID': f'{api_key}'}
     response = requests.get(url, headers=headers)
     return response.json()
 
 def get_anime_detail(anime_id):
     api_key = get_api_key('mal_api')
     if not api_key:
         return {'error': 'API key not found'}
-    url = f"https://api.myanimelist.net/v2/anime/{anime_id}"
-    headers = {'Authorization': f'Bearer {api_key}'}
+    url = f"https://api.myanimelist.net/v2/anime/{anime_id}?fields=id,title,main_picture,alternative_titles,start_date,end_date,synopsis,rank,popularity,status,num_episodes,rating,pictures,background,related_anime"
+    headers = {'X-MAL-CLIENT-ID': f'{api_key}'}
     response = requests.get(url, headers=headers)
     return response.json()

app/services/route_service.py

@@ -0,0 +1,46 @@
+import datetime
+import os
+from flask import request, jsonify, abort, g
+from functools import wraps
+from flask_login import current_user
+
+def login_or_api_key_required(f):
+    @wraps(f)
+    def decorated_function(*args, **kwargs):
+        # Check if user is logged in
+        if not current_user.is_authenticated:
+            # If not logged in, check for API key
+            api_key = request.headers.get('x_api_key')
+            if not api_key or api_key != 'your_api_key_here':  # Validate API key # TBD temporary to allow cron. Will need logic to provision api keys from db, and later on pass to cron
+                abort(403, description="Access denied: No valid API key or login session")
+        return f(*args, **kwargs)
+    return decorated_function
+
+import pkg_resources
+
+def get_installed_packages():
+    packages = {}
+    for dist in pkg_resources.working_set: # Filter to include only packages installed from GitHub
+        packages[dist.project_name] = dist.version
+    return packages
+
+def list_files(path):
+    # Check if the path is valid and is a directory
+    if not os.path.isdir(path):
+        return jsonify({'error': 'Invalid directory path'}), 400
+
+    # List all files in the directory and their details
+    try:
+        files = []
+        for filename in os.listdir(path):
+            file_path = os.path.join(path, filename)
+            if os.path.isfile(file_path):
+                file_info = os.stat(file_path)
+                files.append({
+                    'name': filename,
+                    'size_mb': round(file_info.st_size / (1024 * 1024), 2),  # Convert size to MB and round to 2 decimal places
+                    'last_modified': datetime.fromtimestamp(file_info.st_mtime).strftime('%Y-%m-%d %H:%M:%S')  # Format the date
+                })
+        return jsonify({'files': files})
+    except Exception as e:
+        return jsonify({'error': str(e)}), 500