Skip to content

Commit

Permalink
improve docs for running lightning locally (#2499)
Browse files Browse the repository at this point in the history
  • Loading branch information
midigofrank authored Sep 18, 2024
1 parent 36e11ac commit 6ef345c
Show file tree
Hide file tree
Showing 6 changed files with 181 additions and 50 deletions.
5 changes: 5 additions & 0 deletions DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,11 @@ permissions are needed for the github app:
| Secrets | Read and Write |
| Workflows | Read and Write |

Ensure you set the following URLs:
* **Homepage URL:** `<app_url_here>`
* **Callback URL for authorizing users:** `<app_url_here>/oauth/github/callback` (Do NOT check the two checkboxes in this section requesting Device Flow and OAuth.)
* **Setup URL for Post installation:** `<app_url_here>/setup_vcs` (Check the box for **Redirect on update**)

These envrionment variables will need to be set in order to configure the github
app:

Expand Down
47 changes: 1 addition & 46 deletions Dockerfile-dev
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ WORKDIR /app

# install hex + rebar
RUN mix local.hex --force && \
mix local.rebar --force
mix local.rebar --force

# set build ENV

Expand Down Expand Up @@ -72,48 +72,3 @@ ARG PORT
EXPOSE ${PORT}

CMD ["mix", "phx.server"]

################################################################################

# TODO push the builder layer into a 'prod' layer, and prepend it with a 'builder'
# layer that does the releases build

FROM ${BUILDER_IMAGE} as builder

# compile assets
RUN mix assets.deploy

# Compile the release
COPY lib lib

RUN mix compile

# Changes to config/runtime.exs don't require recompiling the code
COPY config/runtime.exs config/

COPY rel rel
RUN mix release

# start a new build stage so that the final image will only contain
# the compiled release and other runtime necessities
FROM ${RUNNER_IMAGE}

RUN apt-get update -y && apt-get install -y libstdc++6 openssl libncurses5 locales \
&& apt-get clean && rm -f /var/lib/apt/lists/*_*

# Set the locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen

ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

WORKDIR "/app"
RUN chown nobody /app

# Only copy the final release from the build stage
COPY --from=builder --chown=nobody:root /app/_build/prod/rel/lightning ./

USER nobody

CMD ["/app/bin/server"]
32 changes: 32 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -438,6 +438,38 @@ docker compose run --rm web mix ecto.migrate
docker compose up
```

#### Apple Silicon

When running `docker compose up` on Apple Silicon (M1, M2, M3), you might encounter the following error:

```
> [web 20/20] RUN npm install --prefix assets:
0.091 rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2
0.091 Trace/breakpoint trap
------
failed to solve: process "/bin/sh -c npm install --prefix assets" did not complete successfully: exit code: 133
```

You can solve this by setting the default docker platform to `linux/amd64`, i.e. `DOCKER_DEFAULT_PLATFORM=linux/amd64`.
You can read more on this here: https://stackoverflow.com/questions/71040681/qemu-x86-64-could-not-open-lib64-ld-linux-x86-64-so-2-no-such-file-or-direc

You might also run into:

```
[notice] Application ssl exited: exited in: :ssl_app.start(:normal, [])
** (EXIT) an exception was raised:
** (ArgumentError) could not call Module.put_attribute/3 because the module Lightning.MixProject is already compiled
(elixir 1.16.2) lib/module.ex:2360: Module.assert_not_readonly!/2
(elixir 1.16.2) lib/module.ex:2041: Module.__put_attribute__/5
(ssl 11.1.4) ssl_app.erl:35: :ssl_app.stop/1
(kernel 9.2.4) application_master.erl:293: :application_master.start_it_old/4
** (MatchError) no match of right hand side value: {:error, {:ssl, {:bad_return, {{:ssl_app, :start, [:normal, []]}, {:EXIT, {%ArgumentError{message: "could not call Module.put_attribute/3 because the module Lightning.MixProject is already compiled"}, [{Module, :assert_not_readonly!, 2, [file: ~c"lib/module.ex", line: 2360]}, {Module, :__put_attribute__, 5, [file: ~c"lib/module.ex", line: 2041]}, {:ssl_app, :stop, 1, [file: ~c"ssl_app.erl", line: 35]}, {:application_master, :start_it_old, 4, [file: ~c"application_master.erl", line: 293]}]}}}}}}
```

You can resolve this by setting `ERL_FLAGS="+JPperf true"` env to the failing stage.
You can follow this thread on our community forum for more info: https://community.openfn.org/t/lightning-prebuilt-images-throw-no-matching-manifest-for-linux-arm64-v8-in-the-manifest-list-entries/465/15

### Problems with Rambo

When running `mix compile.rambo` on Apple Silicon (an Apple M1/M2, `macarm`,
Expand Down
140 changes: 140 additions & 0 deletions RUNNINGLOCAL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Running Lightning Locally

This guide provides instructions for running Lightning locally, either by
installing dependencies on your machine or using Docker.

## By Installing Dependencies

### Setup

#### Postgres
Requires `postgres 15`.
When running in `dev` mode, the app will use the following credentials to authenticate:
- `PORT`: `5432`
- `USER`: `postgres`
- `PASSWORD`: `postgres`
- `DATABASE`: `lightning_dev`

This can however be overriden by specifying a `DATABASE_URL` env var.
e.g. `DATABASE_URL=postgresql://postgres:postgres@localhost:5432/lightning_dev`

We recommend that you use docker for running postgres as you'll get an exact version that we use:

```sh
docker volume create lightning-postgres-data

docker create \
--name lightning-postgres \
--mount source=lightning-postgres-data,target=/var/lib/postgresql/data \
--publish 5432:5432 \
-e POSTGRES_PASSWORD=postgres \
postgres:15.3-alpine

docker start lightning-postgres
```

#### Elixir, NodeJS
We use [asdf](https://github.com/asdf-vm/asdf) to configure our local
environments. Included in the repo is a `.tool-versions` file that is read by
asdf in order to dynamically make the specified versions of Elixir, Erlang and NodeJs
available. You'll need asdf plugins for
[Erlang](https://github.com/asdf-vm/asdf-erlang),
[NodeJs](https://github.com/asdf-vm/asdf-nodejs)
[Elixir](https://github.com/asdf-vm/asdf-elixir) and
[k6](https://github.com/grimoh/asdf-k6).

#### Libsodium
We use [libsodium](https://doc.libsodium.org/) for encoding values as required
by the
[Github API](https://docs.github.com/en/rest/guides/encrypting-secrets-for-the-rest-api).
You'll need to install `libsodium` in order for the application to compile.

For Mac Users:

```sh
brew install libsodium
```

For Debian Users:

```sh
sudo apt-get install libsodium-dev
```

You can find more on
[how to install libsodium here](https://doc.libsodium.org/installation)

#### Compilation and Assets

```sh
asdf install # Install language versions
mix local.hex
mix deps.get
mix local.rebar --force
[[ $(uname -m) == 'arm64' ]] && CPATH=/opt/homebrew/include LIBRARY_PATH=/opt/homebrew/lib mix deps.compile enacl # Force compile enacl if on M1
[[ $(uname -m) == 'arm64' ]] && mix compile.rambo # Force compile rambo if on M1
mix lightning.install_runtime
mix lightning.install_schemas
mix lightning.install_adaptor_icons
mix ecto.create
mix ecto.migrate
npm install --prefix assets
```

In case you encounter errors running any of these commands, see the [troubleshooting guide](README.md#troubleshooting) for
known errors.

### Running the App
To start the lightning server:

```sh
mix phx.server
```

Once the server has started, head to [`localhost:4000`](http://localhost:4000)
in your browser.

By default, the `worker` is started when run `mix phx.server` in `dev` mode. In case you
don't want to have your worker started in `dev`, set `RTM=false`:

```sh
RTM=false mix phx.server
```

## Using Docker

There is an existing `docker-compose.yaml` file in the project's root which has all the
services required. To start your services:

```sh
docker compose up
```

There 2 docker files in the root, `Dockerfile` builds the app in `prod` mode while `Dockerfile-dev`
runs it in `dev` mode. It is important to note that `mix commands` do not work in the `prod` images.

For exmaple, to run migrations in `dev` mode you run:

```sh
docker compose run --rm web mix ecto.migrate
```

While in `prod` mode:

```sh
docker compose run --rm web /app/bin/lightning eval "Lightning.Release.migrate()"
```

### Configuring the Worker

By default, lightning starts the `worker` when running in `dev`. This can also be configured using
`RTM` env var. In case you don't want the hassle of configuring the worker in `dev`, you can just
remove/comment out the `worker` service from the `docker-compose.yaml` file because lightning will
start it for you.

[Learn more about configuring workers](WORKERS.md)

### Problems with Apple Silicon

You might run into some errors when running the docker containers on Apple Silicon.
[We have documented the known ones here](README.md#problems-with-docker)
6 changes: 2 additions & 4 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ x-lightning: &default-app
build:
dockerfile: Dockerfile-dev
context: '.'
target: '${MIX_ENV:-dev}'
args:
- 'MIX_ENV=${MIX_ENV:-dev}'
- 'NODE_ENV=${NODE_ENV:-development}'
Expand All @@ -25,7 +24,7 @@ services:
- POSTGRES_USER=${POSTGRES_USER:-postgres}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
- POSTGRES_DB=${POSTGRES_DB:-lightning_dev}
image: 'postgres:14.2-alpine'
image: 'postgres:15.3-alpine'
restart: '${DOCKER_RESTART_POLICY:-unless-stopped}'
stop_grace_period: '3s'
volumes:
Expand Down Expand Up @@ -62,8 +61,7 @@ services:
web:
condition: service_healthy
restart: true
command:
['pnpm', 'start:prod', '-l', 'ws://web:${URL_PORT-4000}/worker']
command: ['pnpm', 'start:prod', '-l', 'ws://web:${URL_PORT-4000}/worker']
restart: '${DOCKER_RESTART_POLICY:-unless-stopped}'
stop_grace_period: '3s'
expose:
Expand Down
1 change: 1 addition & 0 deletions mix.exs
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,7 @@ defmodule Lightning.MixProject do
logo: "priv/static/images/square-logo.png",
extras: [
"README.md": [title: "Lightning"],
"RUNNINGLOCAL.md": [title: "Running Locally"],
"DEPLOYMENT.md": [title: "Deployment"],
"benchmarking/README.md": [
title: "Benchmarking",
Expand Down

0 comments on commit 6ef345c

Please sign in to comment.