improve docs for running lightning locally (#2499)

DEPLOYMENT.md

@@ -72,6 +72,11 @@ permissions are needed for the github app:
 | Secrets      | Read and Write |
 | Workflows    | Read and Write |
 
+Ensure you set the following URLs:
+* **Homepage URL:** `<app_url_here>`
+* **Callback URL for authorizing users:** `<app_url_here>/oauth/github/callback` (Do NOT check the two checkboxes in this section requesting Device Flow and OAuth.)
+* **Setup URL for Post installation:** `<app_url_here>/setup_vcs` (Check the box for **Redirect on update**)
+
 These envrionment variables will need to be set in order to configure the github
 app:
 

Dockerfile-dev

@@ -37,7 +37,7 @@ WORKDIR /app
 
 # install hex + rebar
 RUN mix local.hex --force && \
-  mix local.rebar --force
+    mix local.rebar --force
 
 # set build ENV
 
@@ -72,48 +72,3 @@ ARG PORT
 EXPOSE ${PORT}
 
 CMD ["mix", "phx.server"]
-
-################################################################################
-
-# TODO push the builder layer into a 'prod' layer, and prepend it with a 'builder'
-# layer that does the releases build
-
-FROM ${BUILDER_IMAGE} as builder
-
-# compile assets
-RUN mix assets.deploy
-
-# Compile the release
-COPY lib lib
-
-RUN mix compile
-
-# Changes to config/runtime.exs don't require recompiling the code
-COPY config/runtime.exs config/
-
-COPY rel rel
-RUN mix release
-
-# start a new build stage so that the final image will only contain
-# the compiled release and other runtime necessities
-FROM ${RUNNER_IMAGE}
-
-RUN apt-get update -y && apt-get install -y libstdc++6 openssl libncurses5 locales \
-  && apt-get clean && rm -f /var/lib/apt/lists/*_*
-
-# Set the locale
-RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
-
-ENV LANG en_US.UTF-8
-ENV LANGUAGE en_US:en
-ENV LC_ALL en_US.UTF-8
-
-WORKDIR "/app"
-RUN chown nobody /app
-
-# Only copy the final release from the build stage
-COPY --from=builder --chown=nobody:root /app/_build/prod/rel/lightning ./
-
-USER nobody
-
-CMD ["/app/bin/server"]

README.md

@@ -438,6 +438,38 @@ docker compose run --rm web mix ecto.migrate
 docker compose up
 ```
 
+#### Apple Silicon
+
+When running `docker compose up` on Apple Silicon (M1, M2, M3), you might encounter the following error:
+
+```
+> [web 20/20] RUN npm install --prefix assets:
+0.091 rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2
+0.091  Trace/breakpoint trap
+------
+failed to solve: process "/bin/sh -c npm install --prefix assets" did not complete successfully: exit code: 133
+```
+
+You can solve this by setting the default docker platform to `linux/amd64`, i.e. `DOCKER_DEFAULT_PLATFORM=linux/amd64`.
+You can read more on this here: https://stackoverflow.com/questions/71040681/qemu-x86-64-could-not-open-lib64-ld-linux-x86-64-so-2-no-such-file-or-direc
+
+You might also run into:
+
+```
+[notice] Application ssl exited: exited in: :ssl_app.start(:normal, [])
+ ** (EXIT) an exception was raised:
+     ** (ArgumentError) could not call Module.put_attribute/3 because the module Lightning.MixProject is already compiled
+         (elixir 1.16.2) lib/module.ex:2360: Module.assert_not_readonly!/2
+         (elixir 1.16.2) lib/module.ex:2041: Module.__put_attribute__/5
+         (ssl 11.1.4) ssl_app.erl:35: :ssl_app.stop/1
+         (kernel 9.2.4) application_master.erl:293: :application_master.start_it_old/4
+
+** (MatchError) no match of right hand side value: {:error, {:ssl, {:bad_return, {{:ssl_app, :start, [:normal, []]}, {:EXIT, {%ArgumentError{message: "could not call Module.put_attribute/3 because the module Lightning.MixProject is already compiled"}, [{Module, :assert_not_readonly!, 2, [file: ~c"lib/module.ex", line: 2360]}, {Module, :__put_attribute__, 5, [file: ~c"lib/module.ex", line: 2041]}, {:ssl_app, :stop, 1, [file: ~c"ssl_app.erl", line: 35]}, {:application_master, :start_it_old, 4, [file: ~c"application_master.erl", line: 293]}]}}}}}}
+```
+
+You can resolve this by setting `ERL_FLAGS="+JPperf true"` env to the failing stage.
+You can follow this thread on our community forum for more info: https://community.openfn.org/t/lightning-prebuilt-images-throw-no-matching-manifest-for-linux-arm64-v8-in-the-manifest-list-entries/465/15
+
 ### Problems with Rambo
 
 When running `mix compile.rambo` on Apple Silicon (an Apple M1/M2, `macarm`,

RUNNINGLOCAL.md

@@ -0,0 +1,140 @@
+# Running Lightning Locally
+
+This guide provides instructions for running Lightning locally, either by
+installing dependencies on your machine or using Docker.
+
+## By Installing Dependencies
+
+### Setup
+
+#### Postgres
+Requires `postgres 15`.
+When running in `dev` mode, the app will use the following credentials to authenticate:
+- `PORT`: `5432`
+- `USER`: `postgres`
+- `PASSWORD`: `postgres`
+- `DATABASE`: `lightning_dev`
+
+This can however be overriden by specifying a `DATABASE_URL` env var.
+e.g. `DATABASE_URL=postgresql://postgres:postgres@localhost:5432/lightning_dev`
+
+We recommend that you use docker for running postgres as you'll get an exact version that we use:
+
+```sh
+docker volume create lightning-postgres-data
+
+docker create \
+  --name lightning-postgres \
+  --mount source=lightning-postgres-data,target=/var/lib/postgresql/data \
+  --publish 5432:5432 \
+  -e POSTGRES_PASSWORD=postgres \
+  postgres:15.3-alpine
+
+docker start lightning-postgres
+```
+
+#### Elixir, NodeJS
+We use [asdf](https://github.com/asdf-vm/asdf) to configure our local
+environments. Included in the repo is a `.tool-versions` file that is read by
+asdf in order to dynamically make the specified versions of Elixir, Erlang and NodeJs
+available. You'll need asdf plugins for
+[Erlang](https://github.com/asdf-vm/asdf-erlang),
+[NodeJs](https://github.com/asdf-vm/asdf-nodejs)
+[Elixir](https://github.com/asdf-vm/asdf-elixir) and
+[k6](https://github.com/grimoh/asdf-k6).
+
+#### Libsodium
+We use [libsodium](https://doc.libsodium.org/) for encoding values as required
+by the
+[Github API](https://docs.github.com/en/rest/guides/encrypting-secrets-for-the-rest-api).
+You'll need to install `libsodium` in order for the application to compile.
+
+For Mac Users:
+
+```sh
+brew install libsodium
+```
+
+For Debian Users:
+
+```sh
+sudo apt-get install libsodium-dev
+```
+
+You can find more on
+[how to install libsodium here](https://doc.libsodium.org/installation)
+
+#### Compilation and Assets
+
+```sh
+asdf install  # Install language versions
+mix local.hex
+mix deps.get
+mix local.rebar --force
+[[ $(uname -m) == 'arm64' ]] && CPATH=/opt/homebrew/include LIBRARY_PATH=/opt/homebrew/lib mix deps.compile enacl # Force compile enacl if on M1
+[[ $(uname -m) == 'arm64' ]] && mix compile.rambo # Force compile rambo if on M1
+mix lightning.install_runtime
+mix lightning.install_schemas
+mix lightning.install_adaptor_icons
+mix ecto.create
+mix ecto.migrate
+npm install --prefix assets
+```
+
+In case you encounter errors running any of these commands, see the [troubleshooting guide](README.md#troubleshooting) for
+known errors.
+
+### Running the App
+To start the lightning server:
+
+```sh
+mix phx.server
+```
+
+Once the server has started, head to [`localhost:4000`](http://localhost:4000)
+in your browser.
+
+By default, the `worker` is started when run `mix phx.server` in `dev` mode. In case you
+don't want to have your worker started in `dev`, set `RTM=false`:
+
+```sh
+RTM=false mix phx.server
+```
+
+## Using Docker
+
+There is an existing `docker-compose.yaml` file in the project's root which has all the
+services required. To start your services:
+
+```sh
+docker compose up
+```
+
+There 2 docker files in the root, `Dockerfile` builds the app in `prod` mode while `Dockerfile-dev`
+runs it in `dev` mode. It is important to note that `mix commands` do not work in the `prod` images.
+
+For exmaple, to run migrations in `dev` mode you run:
+
+```sh
+docker compose run --rm web mix ecto.migrate
+```
+
+While in `prod` mode:
+
+```sh
+docker compose run --rm web /app/bin/lightning eval "Lightning.Release.migrate()"
+```
+
+### Configuring the Worker
+
+By default, lightning starts the `worker` when running in `dev`. This can also be configured using
+`RTM` env var. In case you don't want the hassle of configuring the worker in `dev`, you can just
+remove/comment out the `worker` service from the `docker-compose.yaml` file because lightning will
+start it for you.
+
+[Learn more about configuring workers](WORKERS.md)
+
+### Problems with Apple Silicon
+
+You might run into some errors when running the docker containers on Apple Silicon.
+[We have documented the known ones here](README.md#problems-with-docker)

docker-compose.yml

@@ -2,7 +2,6 @@ x-lightning: &default-app
   build:
     dockerfile: Dockerfile-dev
     context: '.'
-    target: '${MIX_ENV:-dev}'
     args:
       - 'MIX_ENV=${MIX_ENV:-dev}'
       - 'NODE_ENV=${NODE_ENV:-development}'
@@ -25,7 +24,7 @@ services:
       - POSTGRES_USER=${POSTGRES_USER:-postgres}
       - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
       - POSTGRES_DB=${POSTGRES_DB:-lightning_dev}
-    image: 'postgres:14.2-alpine'
+    image: 'postgres:15.3-alpine'
     restart: '${DOCKER_RESTART_POLICY:-unless-stopped}'
     stop_grace_period: '3s'
     volumes:
@@ -62,8 +61,7 @@ services:
       web:
         condition: service_healthy
         restart: true
-    command:
-      ['pnpm', 'start:prod', '-l', 'ws://web:${URL_PORT-4000}/worker']
+    command: ['pnpm', 'start:prod', '-l', 'ws://web:${URL_PORT-4000}/worker']
     restart: '${DOCKER_RESTART_POLICY:-unless-stopped}'
     stop_grace_period: '3s'
     expose:

mix.exs

@@ -195,6 +195,7 @@ defmodule Lightning.MixProject do
       logo: "priv/static/images/square-logo.png",
       extras: [
         "README.md": [title: "Lightning"],
+        "RUNNINGLOCAL.md": [title: "Running Locally"],
         "DEPLOYMENT.md": [title: "Deployment"],
         "benchmarking/README.md": [
           title: "Benchmarking",