Architecture

frappe-manager Python Package

• This package introduces fm command
• This package serves as the entry point for the Frappe-Manager project and provides the following key functionalities:

1. Metadata Generation: Generates essential metadata required for setting up a Bench environment, including benchname, admin password, and the list of Frappeverse apps to be installed.

2. Docker Compose Configuration: Customizes and generates a Docker Compose project using the metadata as input for the entrypoint script. This project contains various services to support Frappe development.

3. Docker Compose Management: Wraps the Docker Compose CLI to manage the Docker Compose project, allowing for easy control and orchestration of containers for the development environment.

4. Environment Management Commands: Provides commands to manage the FrappeManager environment, streamlining common development tasks.

5. CLI Directory Creation: Creates a directory at ~/frappe to store frappe-manager managed benches. This directory contains the following subdirectories:

   • logs: Stores cli logs.
   • sites: Stores the Docker Compose project for benches.
   • migration: Stores migration backups.
   • archived: Stores failed migrated benches.
   • services: Stores the Docker Compose project for services.

Services Compose Project

• Located at ~/frappe/services.

• Manages these two services:

  1. global-db:
  • This service serves as the central database for all benches. All bench databases are stored here.
  • Uses mariadb:10.6 image.
  • The database credentials and configuration are stored in Docker secrets.
  • The following directories are mounted for this service:

  Source	Destination	Purpose
  fm-global-db-data	/var/lib/mysql	Database data (OSX)
  ./mariadb/data	/var/lib/mysql	Database data (Linux)
  ./mariadb/conf	/etc/mysql	Configuration files
  ./mariadb/logs	/var/log/mysql	Log files

  2. global-nginx-proxy:
  • This service functions as a reverse proxy for all benches and exposes ports 80 and 443 to host.
  • Uses jwilder/nginx-proxy image.
  • The following directories are mounted for this service:

  Source	Destination	Purpose
  ./nginx-proxy/certs	/etc/nginx/certs	SSL certificates
  ./nginx-proxy/dhparam	/etc/nginx/dhparam	Diffie-Hellman parameters
  ./nginx-proxy/confd	/etc/nginx/conf.d	Config files generated by docker gen
  ./nginx-proxy/htpasswd	/etc/nginx/htpasswd	HTTP Basic Authentication
  ./nginx-proxy/vhostd	/etc/nginx/vhost.d	vhost configuration
  ./nginx-proxy/html	/usr/share/nginx/html	Static HTML files
  ./nginx-proxy/logs	/var/log/nginx	Log files
  ./nginx-proxy/run	/var/run	PID files
  ./nginx-proxy/ssl	/usr/share/nginx/ssl	SSL service related files
  ./nginx-proxy/cache	/var/cache/nginx	caching
  /var/run/docker.sock	/tmp/docker.sock:ro	Docker socket

Implementation differences between OSX and Linux

OSX

The user directive in Docker Compose is not utilized in OSX because the osxfs file system driver automatically handles UID/GID remapping. This ensures that files created within the container maintain the correct permissions when accessed from the host, and vice versa.

global-db: For data persistence, a Docker volume mount (fm-global-db-data) is used instead of a bind mount(causes issues on macOS).

Linux

On Linux, the user directive is used to specify the user (and group) under which the container runs. This is often necessary to prevent permission issues on Linux.

global-db: A bind mount (./mariadb/data) is used for data persistence.

Bench Compose Project

• Bench compose compose project compose off bench services, worker services and Admin Tools services.
• Bench configuration can be changed using bench_config.toml configuration file.
• Workers services are set to active by default and remain operational as long as the bench is running.
• Admin tools services are enabled and disabled based on bench_config.toml.

Bench services

• Frappe (custom build):

  • Configures the server environment, including setting up pyenv and server-specific configurations.
  • Creates a Bench environment for Frappe app development.
  • Installs required Frappeverse apps.
  • Creates and manages Frappe sites with associated databases.
  • Initiates Supervisor as the init process, containing a service for starting the Frappe development server.

• Nginx (custom build):

  • Extends the features of the official Nginx Docker image.
  • Incorporates input from the frappe-manager Python package to create bench-specific Nginx configurations from templates.

• redis-cache, redis-queue, redis-socketio: They set up Redis servers for caching, queuing, and real-time communication.

• socketio: It sets up a Socket.IO server for real-time communication.

• schedule: It sets up a scheduler for running background tasks.

Workers services

• long worker: Worker for frappe long queue.
• short worker: Worker for frappe short queue.

Admin Tools services

• Adminer: A fully-featured database management tool, serving as an alternative to phpMyAdmin for database management.
• Mailhog (custom build):
  • Custom build provided arm64 supported image.
  • An email testing tool with a fake SMTP server underneath, enabling email testing during development.

CLI logs

• Stored in /home/<user>/frappe/logs/fm.log
• Logs contains:
  • every fm command invoked by user.
  • executed docker commands, command output and return code.
  • exception happend during runtime.
• Logs are rotated when the log size hits 10MB, creates files like fm.log.1 when rotate limit is reached and only 3 such files are kept.