Which file/location to point to in nginx config?

[wkulesza]

When running the Datatools UI and server, front-end is served by nginx, but it's unclear on which file of UI controls the dynamic urls and thus, how to setup nginx to point to index.html? Without this, when trying to refresh any url, I'm getting "403 The requested path could not be found" message. The URL adds "/login/", or "/home/4a80f2a2-d3a3-4cd9-af66-99d716d0ad44" or other similar suffixes.
I'm currently on:
UI Version: 1391fb
Server version: 2a725e

[miles-grant-ibigroup]

How are you running the UI? depending on the command you use the answer for this changes

[wkulesza]

Im using (sometimes replacing production with test):

mastarm build --env production && serve -p 9966

[wkulesza]

If that helps @miles-grant-ibigroup this is the ui that is then served by nginx and it's config is:

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;


    upstream front {
      server datatools-ui:9966;
    }

    upstream back {
      server datatools-server:4000;
    }

    server {
      listen 8081;

      location /api {
        #proxy_set_header X-Real-IP $remote_addr;
        #proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        #proxy_set_header Host $http_host;
        #proxy_set_header X-NginX-Proxy true;

        proxy_pass http://back/api;
        #proxy_redirect off;
      }

      location / {
        proxy_pass http://front/;

      }
    }
}

[wkulesza]

@miles-grant-ibigroup would you have any hints ?

[miles-grant-ibigroup]

I don't think you need to manually set your /api route. Try hosting your datatools server on a separate host.

Merging the 2 together on one endpoint is something I've been struggling with for a while too...

[wkulesza]

@miles-grant-ibigroup when react app is served by Apache it's enough to set the vhost to point all requests to index.html of react app that does the rest of rourting.

'''

<VirtualHost *:8080>
ServerName example.com
DocumentRoot /var/www/httpd/example.com

<Directory "/var/www/httpd/example.com">
...

RewriteEngine On
# Don't rewrite files or directories
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ - [L]
# Rewrite everything else to index.html to allow html5 state links
RewriteRule ^ index.html [L]

'''

I tried doing similar with nginx conf file but with no success;

'''

Any route containing a file extension (e.g. /devicesfile.js)

location ~ ^.+..+$ {
try_files $uri =404;
}

Any route that doesn't have a file extension (e.g. /devices)

location / {
try_files $uri $uri/ /index.html;
}

'''

[miles-grant-ibigroup]

Try first just routing to the datatools server. the server includes an old version of the UI. once you have this routing in place and working, try swapping out just the non-api routes. in our experience we've actually found most helpful to host the datatools server on one server and host the UI separately!

[wkulesza]

Can you elaborate?
What is the point of having ui and server on different servers?

[mvanlaar]

I have here: https://github.com/ColombiaTransit/datatools-ui a config for docker with lighthttpd as the server serving the ui. Traefik routes in my situation the /api requests to the datatools server. I'm hosting the datatools-server in a separate container.

[wkulesza]

@mvanlaar Hi - thanks for this. We're using a docker-compose project that builds datatools-server, datatools-ui, postgis and mongodb and serves all through nginx. So i reckon here, we would need to replace nginx with lighthttpd. How to mix this with Traefik? Do you have any template to use ?

[mvanlaar]

My redacted traefik file:

http:

  ## EXTERNAL ROUTING EXAMPLE - Only use if you want to proxy something manually ##
  routers:
    datatools-ui:
      entryPoints:
        - web        
      rule: "Host(`datatools.example.org`)"      
      service: datatools-ui@docker
      middlewares:
        - datatools-ui@file
    datatools-api:
      entryPoints:
        - web        
      rule: "Host(`datatools.example.org`) && PathPrefix(`/api`)"      
      service: datatools-server@docker
      middlewares:
        - datatools-api@file    
    datatools-graphhopper-api:
      entryPoints:
        - web        
      rule: "Host(`datatools.example.org`) && PathPrefix(`/graphhopper`)"      
      service: graphhopper-api@docker
      middlewares:
        - datatools-graphhopper-api@file    
  # allow self-signed certificates for proxied web services
  serversTransports:
    insecureTransport:
      insecureSkipVerify: true

  ## MIDDLEWARES ##
  middlewares:
    # Only Allow Local networks
    datatools-api:
      chain:
        middlewares:    
          - default-headers
          - real-ip
          - datatools-cors-headers
    datatools-graphhopper-api:
      chain:
        middlewares:    
          - default-headers
          - real-ip
          - datatools-cors-headers
          - cleanup-datatools-request
    datatools-ui:
      chain:
        middlewares:    
          - default-headers
          - real-ip
          - datatools-cors-headers
    datatools-cors-headers:
      headers:
        accessControlAllowMethods:
          - GET
          - OPTIONS
          - POST
          - PUT
        accessControlAllowOriginList:
          - https://datatools.example.org          
        accessControlMaxAge: 100
        addVaryHeader: true
        accessControlAllowHeaders: "*"          
        customRequestHeaders:
          X-Forwarded-Proto: https
    cleanup-datatools-request:
      stripPrefix:
        prefixes:
          - "/graphhopper"    ```

[wkulesza]

Thanks @mvanlaar will give Traefik a try. And you don't have mongodb and postgis in the docker-compose?
Our docker-compose (for all Datatools project) is:
https://github.com/goeuropa/gtfs_editor_ibi_datatools_docker/blob/master/docker-compose.yml

[mvanlaar]

Thanks @mvanlaar will give Traefik a try. And you don't have mongodb and postgis in the docker-compose? Our docker-compose (for all Datatools project) is: https://github.com/goeuropa/gtfs_editor_ibi_datatools_docker/blob/master/docker-compose.yml

Yes i have a docker compose with those services also.

[mvanlaar]

@mvanlaar Hi - thanks for this. We're using a docker-compose project that builds datatools-server, datatools-ui, postgis and mongodb and serves all through nginx. So i reckon here, we would need to replace nginx with lighthttpd. How to mix this with Traefik? Do you have any template to use ?

I use lighthttpd only because i don't need the full webserver of nignx, it serves only static files (ui). So for me the lighthttpd is sufficicient

[wkulesza]

Cool. With my current setup, i'm struggling with 404 The requested path could not be found for all urls that are generated.
Would Traefik solve this? So I can use the current docker-compose but replace the nginx part with traefik docker-compose and then open up the traefik port rather than nginx port in browser? Maybe we could discuss this off issues, not to clutter this ?

[mvanlaar]

My docker UI includes the webserver (lighthttpd) part. Look in the dockerfile and actions of the above repo link to see how this is build. I've got 2 main docker containers (server - with separate docker for postegres and mongo) and the ui docker. traefik controlls the access to the ui and rewirtes only the /api part. You can include this also in the lighthttpd part i think.

[miles-grant-ibigroup]

Can you elaborate?
What is the point of having ui and server on different servers?

I didn't make this decision, but I believe it was done so that the ui and server could be deployed separately

[mvanlaar]

It's about using the cloud at its best. Ui only needs cheap storage like s3, github pages etc. to host the website part. The processing parts are done at the server.

[changtung]

In this config file:

i would like to clone git repository with datatools-ui code. However i don't understand exactly where source of datatools-ui is located. I need this since i need to run working version of datatools-ui with access to datatools-server. Or where do i set datatools-server host ?

[mvanlaar]

The github action on my repo creates the config file witch is included in the build of the application. The .js and .css files are generated in the dist folder after the build. These are copied with the index.html

It's a 2 stage build to only include only what's needed and keep the container small. The /static/ folder i use for including the logo etc.

[wkulesza]

@mvanlaar does your setup allow to refresh any URL that is created (/i.e. /home /login or /admin?) and able to open l/admin/users?).
We're struggling with being able to open any link other than the main page.

[brodyFlannigan]

Following this issue. I used the GoEuropa docker setup which works pretty much flawlessly in terms of actually getting stuff set up (thanks @wkulesza for getting that set up) but I'm struggling with 404s being returned for any calls to the API, unfortunately rendering the whole thing useless.
@mvanlaar if you could write detailed instructions on how to get your setup going with Traefik and stuff (i.e. instructions for starting from scratch and how to set up the Traefik config) that'd be greatly appreciated.
If anyone has had any luck with the GoEuropa setup so that the API doesn't return 404s please share how you did it!