Skip to content
Lou Wolford edited this page Mar 27, 2017 · 12 revisions

Using Celery

Retrieving Activity Metadata

It is stated in the xAPI spec that, "If an Activity IRI is an IRL, an LRS SHOULD attempt to GET that IRL, and include in HTTP headers: "Accept: application/json, /". This SHOULD be done as soon as practical after the LRS first encounters the Activity id. Upon loading JSON which is a valid Activity Definition from an IRL used as an Activity id, an LRS SHOULD incorporate the loaded definition into its internal definition for that Activity, while preserving names or definitions not included in the loaded definition."

The ADL LRS will now retrieve activity metadata, void statements, and check LRS hooks via celery. This speeds up the processing of incoming statements, especially those that are in large batches, and gives the client a faster response.

Voiding Statements

In case batch statements don't come in the correct order when voiding statements, the LRS will wait until all statements from that batch are stored, then will start worker processes to attempt to void the statements that were specified instead of returning a 404 error when trying to void statements sequentially.

Setup

The LRS will be using Celery as an asynchronous task queue and RabbitMQ as its message broker to retrieve the activity metadata. This will be running completely independent of Django. The libraries required for this should already be installed in your environment (amqp and celery) from the requirements.txt document.

RabbitMQ Setup

For details and more in-depth documentation, visit the celery docs.

  1. admin:~$ sudo apt-get install rabbitmq-server
  2. admin:~$ sudo rabbitmqctl add_user <username_for_rabbitmq> <password_for_rabbitmq>
  3. admin:~$ sudo rabbitmqctl add_vhost <vhost_name>
  4. admin:~$ sudo rabbitmqctl set_permissions -p <vhost_name> <username_for_rabbitmq> ".*" ".*" ".*"

Celery Setup

For details and more in-depth documentation, visit the celery docs.

  1. Configure /path/to/adl_lrs/settings.ini

    [ampq]
    USERNAME: <username_for_rabbitmq>
    PASSWORD: <password_for_rabbitmq>
    HOST: localhost
    PORT: 5672
    VHOST: <vhost_name>
    
  2. Create supervisord.conf file in /path/to/ADL_LRS/

    [unix_http_server]
    file=/tmp/supervisor.sock                       ; path to your socket file
    
    [supervisord]
    logfile=/path/to/logs/supervisord/supervisord.log ; CHANGE THIS PATH
    childlogdir=/path/to/logs/supervisord/            ; CHANGE THIS PATH  
    logfile_maxbytes=50MB                           ; maximum size of logfile before rotation
    logfile_backups=10                              ; number of backed up logfiles
    loglevel=info                                  ; info, debug, warn, trace
    pidfile=/var/run/supervisord.pid                ; pidfile location
    minfds=1024                                     ; number of startup file descriptors
    minprocs=200                                    ; number of process descriptors
    
    [rpcinterface:supervisor]
    supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
    
    [supervisorctl]
    serverurl=unix:///tmp/supervisor.sock         ; use a unix:// URL  for a unix socket
    
    [program:celery]
    ; Set full path to celery program if using virtualenv
    command=/path/to/env/bin/celery worker -A lrs --loglevel=INFO ; CHANGE THIS PATH
    
    
    directory=/path/to/ADL_LRS ; CHANGE THIS PATH
    numprocs=1
    stdout_logfile=/path/to/logs/celery/celery-worker.log        ; CHANGE THIS PATH
    stderr_logfile=/path/to/logs/celery/celery-worker-errors.log ; CHANGE THIS PATH
    autostart=true
    autorestart=true
    startsecs=10
    
    ; Need to wait for currently executing tasks to finish at shutdown.
    ; Increase this if you have very long running tasks.
    stopwaitsecs = 600
    
    ; When resorting to send SIGKILL to the program to terminate it
    ; send SIGKILL to its whole process group instead,
    ; taking care of its children as well.
    killasgroup=true
    
    ; if rabbitmq is supervised, set its priority higher
    ; so it starts first
    priority=998
    
    
  3. Create upstart script in /etc/init/ (our file will be called celerylrs.conf). You will need to use sudo for admin privileges.

    description    "supervisor for lrs-celery"
    start on runlevel    [2345]
    stop on runlevel    [!2345]
    
    respawn
    
    setuid <Name of system user LRS is running under>
    chdir /path/to/ADL_LRS
    exec /path/to/env/bin/supervisord --nodaemon
    

    You can stop start/stop the celery tasks by typing sudo {start|stop|restart} celerylrs

    NOTE: If you're using Ubuntu 15+ it uses systemd instead of upstart by default. Follow these instructions for a systemd script

    Create systemd script in /lib/systemd/system/ (our file will be called celerylrs.service). You will need to use sudo for admin privileges.

    [Unit]
    Description=Supervisor to run celery for the LRS
    
    [Service]
    User=<Name of system user LRS is running under>
    WorkingDirectory=/path/to/ADL_LRS
    ExecStart=/path/to/env/bin/supervisord --nodaemon
    Restart=always
    
    [Install]
    WantedBy=multi-user.target
    

    Then run sudo systemctl daemon-reload and sudo systemctl enable celerylrs.service.

    You can stop start/stop the celery tasks by typing sudo systemctl {start|stop|restart} celerylrs