Skip to content

Commit

Permalink
Completes the documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
prasadtalasila committed Jun 30, 2023
1 parent 36d69dc commit f444898
Show file tree
Hide file tree
Showing 17 changed files with 659 additions and 189 deletions.
3 changes: 1 addition & 2 deletions deploy/vagrant/single-machine/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,11 @@ A dummy **foo.com** URL has been used for illustration. Please change this to yo

Please follow these steps to make this work in your local environment.

1. Create [**dtaas** Vagrant box](../make_boxes/dtaas/README.md). Copy _vagrant_ SSH private key here. This shall be useful for logging into the vagrant machine create for single-machine deployment. You would have created an SSH key pair - _vagrant_ and _vagrant.pub_. The _vagrant_ is the private SSH key and is needed for the next steps.
1. Create [**dtaas** Vagrant box](../make_boxes/dtaas/README.md). Copy _vagrant_ SSH private key here. This shall be useful for logging into the vagrant machine created for single-machine deployment. You would have created an SSH key pair - _vagrant_ and _vagrant.pub_. The _vagrant_ is the private SSH key and is needed for the next steps.
1. Update the **Vagrantfile**. Fields to update are:
1. Hostname (`node.vm.hostname = "foo.com"`)
1. MAC address (`:mac => "xxxxxxxx"`). This change is required if you have a DHCP server assigning domain names based on MAC address. Otherwise, you can leave this field unchanged.
1. Other adjustments are optional.
1. Copy _vagrant_ SSH private key into the current directory (`deploy/vagrant/single-machine`).
1. Execute the following commands from terminal

```bash
Expand Down
2 changes: 1 addition & 1 deletion deploy/vagrant/two-machine/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ The first step is to define the network identity of the two VMs. For that, you n

Update the **boxes.json**. There are entries one for each server. The fields to update are:

1. `name` - name of server1 (`"name" = "workspaces"`)
1. `name` - name of server1 (`"name" = "dtaas"`)
1. `hostname` - hostname of server1 (`"name" = "foo.com"`)
1. MAC address (`:mac => "xxxxxxxx"`). This change is required if you have a DHCP server assigning domain names based on MAC address. Otherwise, you can leave this field unchanged.
1. `name` - name of server2 (`"name" = "services"`)
Expand Down
5 changes: 5 additions & 0 deletions docs/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

DTaaS is software platform on which you can create and run digital twins. Please see the [features](user/features.md) page to get a sense of the things you can do in DaaS.


??? Question "Are there any Key Performance / Capability Indicators for DTaaS?"

| Key Performance Indicator | Value |
Expand All @@ -28,6 +29,10 @@
| Availability | It is a matter of human resources. If you have human resources to maintain DTaaS round the clock, upwards 95% is easily possible. |


??? Question "Do you provide licensed software like Matlab?"

The licensed software are not available on the software platform. But users have private workspaces which are based on Linux-based xfce Desktop environment. Users can install software in their workspaces. The licensed software installed by one user is not available to another user.


## Digital Twin Models

Expand Down
52 changes: 48 additions & 4 deletions docs/admin/client/CLIENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@

To host DTaaS client website on your server, follow these steps:

- The `build` folder contains all the optimized static files that are ready for deployment.
- Download the **DTaaS-client.zip** from the [releases page](https://github.com/INTO-CPS-Association/DTaaS/releases).
- Inside the `DTaaS-client` directory, there is `site` directory. The `site` directory contains all the optimized static files that are ready for deployment.

- Locate the file `build/env.js` and replace the example values to match your infrastructure. The constructed links will be "`REACT_APP_URL`/`REACT_APP_URL_BASENAME`/`{username}`/`{Endpoint}`". See the definitions below:
- Locate the file `site/env.js` and replace the example values to match your infrastructure. See the definitions below:

```js
window.env = {
Expand Down Expand Up @@ -36,12 +37,55 @@ To host DTaaS client website on your server, follow these steps:
};
```


- Copy the entire contents of the build folder to the root directory of your server where you want to deploy the app. You can use FTP, SFTP, or any other file transfer protocol to transfer the files.

- Make sure your server is configured to serve static files. This can vary depending on the server technology you are using, but typically you will need to configure your server to serve files from a specific directory.

- Once the files are on your server, you should be able to access your app by visiting your server's IP address or domain name in a web browser.
:fontawesome-solid-circle-info: The website depends on Traefik and ML Workspace components to be available. Otherwise, you only get a skeleton non-functional website.
:fontawesome-solid-circle-info: The website depends on **Traefik gateway** and **ML Workspace** components to be available. Otherwise, you only get a skeleton non-functional website.
## Complementary Components
The website requires background services for providing actual functionality. The minimum background service required is atleast one [ML Workspace](https://github.com/ml-tooling/ml-workspace) serving the following routes.
```js
https:foo.com/<username>/lab
https:foo.com/<username>/terminals/main'
https:foo.com/<username>/tools/vnc/?password=vncpassword
https:foo.com/<username>/tools/vscode/
```

The `username` is the user workspace created using ML Workspace docker container. Please follow the instructions in [README](https://github.com/ml-tooling/ml-workspace/blob/main/README.md). You can create as many user workspaces as you want. If you have two users - alice and bob - on your system, then the following the commands in will instantiate the required user workspaces.

```bash
mkdir -p files/alice files/bob files/common

printf "\n\n start the user workspaces"
docker run -d \
-p 8090:8080 \
--name "ml-workspace-alice" \
-v "$(pwd)/files/alice:/workspace" \
-v "$(pwd)/files/common:/workspace/common" \
--env AUTHENTICATE_VIA_JUPYTER="" \
--env WORKSPACE_BASE_URL="alice" \
--shm-size 512m \
--restart always \
mltooling/ml-workspace:0.13.2



docker run -d \
-p 8091:8080 \
--name "ml-workspace-bob" \
-v "$(pwd)/files/bob:/workspace" \
-v "$(pwd)/files/common:/workspace/common" \
--env AUTHENTICATE_VIA_JUPYTER="" \
--env WORKSPACE_BASE_URL="bob" \
--shm-size 512m \
--restart always \
mltooling/ml-workspace:0.13.2
```

Given that multiple services are running at different routes, a reverse proxy is needed to map the background services to external routes. You can use Apache, NGINX, Traefik or any other software to work as reverse proxy.
92 changes: 92 additions & 0 deletions docs/admin/host.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# DTaaS on Linux Operating System

These are installation instructions for running DTaaS application on a Ubuntu Server 20.04 Operating System. The setup requires a machine which can spare 16GB RAM, 8 vCPUs and 50GB Hard Disk space.

A dummy **foo.com** URL has been used for illustration. Please change this to your unique website URL. It is assumed that you are going to serve the application in only HTTPS mode.

Please follow these steps to make this work in your local environment. Download the **DTaaS.zip** from the [releases page](https://github.com/INTO-CPS-Association/DTaaS/releases). Unzip the same into a directory named **DTaaS**. The rest of the instructions assume that your working directory is **DTaaS**.

!!! note
If you only want to test the application and are not setting up a production instance, you can follow the instructions of [trial installation](trial.md).

## Configuration

You need to configure the Traefik gateway, library microservice and react client website.

The first step is to decide on the number of users and their usenames. The traefik gateway configuration has a template for two users. You can modify the usernames in the template to the usernames chosen by you.

### The traefik gateway server

You can run the Run the Traefik gateway server in both and HTTPS HTTPS mode to experience the DTaaS application. The installation guide assumes that you can run the application in HTTPS mode.

The Traefik gateway configuration is at _deploy/config/gateway/fileConfig.yml_. Change `foo.com` to your local hostname and user1/user2 to the usernames chosen by you.

!!! tip

Do not use `http://` or `https://` in _deploy/config/gateway/fileConfig.yml_.

#### Authentication

The dummy username is `foo` and the password is `bar`.
Please change this before starting the gateway.

```bash
rm deploy/config/gateway/auth
touch deploy/config/gateway/auth
htpasswd deploy/config/gateway/auth <first_username>
password: <your password>
```

The user credentials added in _deploy/config/gateway/auth_ should match the usernames in _deploy/config/gateway/fileConfig.yml_.

## Configure lib microservice

The first step in this configuration is to prepare the a filesystem for users. An example file system in `files/` directory. You can rename the top-level user1/user2 to the usernames chosen by you.

Update the _deploy/config/lib_ of the library microservice. The simplest possibility is to use `local` mode with the following example. The filepath is the absolute filepath to `files/` directory.

```env
PORT='4001'
MODE='local'
LOCAL_PATH ='filepath'
LOG_LEVEL='debug'
APOLLO_PATH='/lib'
GRAPHQL_PLAYGROUND='true'
```

## Configure react website

Change the React website configuration in _deploy/config/client/env.js_.

```js
window.env = {
REACT_APP_ENVIRONMENT: 'prod',
REACT_APP_URL: 'https://foo.com/',
REACT_APP_URL_BASENAME: '',
REACT_APP_URL_DTLINK: '/lab',
REACT_APP_URL_LIBLINK: '',
REACT_APP_WORKBENCHLINK_TERMINAL: '/terminals/main',
REACT_APP_WORKBENCHLINK_VNCDESKTOP: '/tools/vnc/?password=vncpassword',
REACT_APP_WORKBENCHLINK_VSCODE: '/tools/vscode/',
REACT_APP_WORKBENCHLINK_JUPYTERLAB: '/lab',
REACT_APP_WORKBENCHLINK_JUPYTERNOTEBOOK: '',
};
```

## Update the installation script

Open `deploy/install.sh` and update user1/user2 to usernames chosen by you.

## Perform the Installation

Go to the DTaaS directory and execute

```sh
source deploy/install.sh
```

You can run this script multiple times until the installation is successful.

## Access the application

Now you should be able to access the DTaaS application at: _https://foo.com_
71 changes: 54 additions & 17 deletions docs/admin/servers/lib/LIB-MS.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
# Host Library Microservice

The **lib microservice** is a simplified file manager providing graphQL API. It has three features:

* provide a listing of directory contents.
* transfer a file to user.
* Source files can either come from local file system or from a gitlab instance.

The library microservice is designed to manage and serve files, functions, and models to users, allowing them to access and interact with various resources.

This document provides instructions for running the library microservice.
This document provides instructions for running a stand alone library microservice.

## Setup the File System

Expand All @@ -11,31 +17,45 @@ The users expect the following file system structure for their reusable assets.
![File System Layout](file-system-layout.png)

There is a skeleton file structure in [DTaaS codebase](https://github.com/INTO-CPS-Association/DTaaS/tree/feature/distributed-demo/files). You can copy and create file system for your users.

## Gitlab setup (optional)

For this microserivce to be functional, a certain directory or gitlab project structure is expected. The microservice expects that the gitlab consisting of one group, DTaaS, and within that group, all of the projects be located, **user1**, **user2**, ... , as well as a **commons** project. Each project corresponds to files of one user.
A sample file structure can be seen in [gitlab dtaas group](https://gitlab.com/dtaas). You can visit the gitlab documentation on [groups](https://docs.gitlab.com/ee/user/group/) for help on the management of gitlab groups.

You can clone the git repositories from the `dtaas` group to get a sample file system structure for the lib microservice.

## Setup Microservice

To set up the lib microservice, follow these steps:

```sh
git clone https://github.com/INTO-CPS-Association/DTaaS.git
cd DTaaS/server/lib
yarn install # Install the required dependencies
```
Download the **lib-microservice.zip** from the [releases page](https://github.com/INTO-CPS-Association/DTaaS/releases).

### Environment Variables
## Configuration setup

To set up the environment variables for the lib microservice, create a new file named _.env_ in the `servers/lib` folder. Then, add the following variables and their respective values. Below you can see and how, with included examples:
The microservices uses `.env` environment files to receive configuration.

To set up the environment variables for the lib microservice, create a new file named _.env_ in the `lib-ms` directory. Then, add the following variables and their respective values. Below you can see and how, with included examples:

```ini
PORT='4001'
MODE='local' or 'gitlab'
LOCAL_PATH='/Users/<Username>/DTaaS/files'
GITLAB_GROUP='dtaas'
LOCAL_PATH ='/Users/<Username>/DTaaS/files'
GITLAB_GROUP ='dtaas'
GITLAB_URL='https://gitlab.com/api/graphql'
TOKEN='123-sample-token'
LOG_LEVEL='debug'
TEST_PATH='/Users/<Username>/DTaaS/servers/lib/test/data/test_assets'
APOLLO_PATH='/lib' or ''
GRAPHQL_PLAYGROUND='false' or 'true'
```

The `LOCAL_PATH` variable is the absolute filepath to the location of the local directory which will be served to users by the Library microservice.

The `GITLAB_URL`, `GITLAB_GROUP` and `TOKEN` are only relevant for `gitlab` mode. The `TOKEN` should be set to your GitLab Group access API token. For more information on how to create and use your access token, [gitlab page](https://docs.gitlab.com/ee/user/group/settings/group_access_tokens.html).

Once you've generated a token, copy it and replace the value of `TOKEN` with your token for the gitlab group, can be found.

Replace the default values the appropriate values for your setup.

**NOTE**:
Expand All @@ -46,19 +66,36 @@ Replace the default values the appropriate values for your setup.
### Start Microservice

```bash
yarn install
yarn build
yarn start
yarn install # Install dependencies for the microservice
yarn build # build the application
yarn start # start the application
```

You can press `Ctl+C` to halt the application. If you wish to run the microservice in the background, use

```bash
nohup yarn start & disown
```

The lib microservice is now running and ready to serve files, functions, and models.

Users can access the library microservice at URL: `http://localhost:<PORT>/lib`.

## Developer Commands

```bash
yarn install # Install dependencies for the microservice
yarn syntax # analyzes source code for potential errors, style violations, and other issues,
yarn build # compile ES6 files into ES5 javascript files and copy all JS files into build/ directory
yarn test -a # run all tests
yarn test -e # run end-to-end tests
yarn test -i # run integration tests
yarn test -u # run unit tests
yarn start # start the application
yarn clean # deletes directories "build", "coverage", and "dist"
```

### Modes of Operation
## Service Endpoint

The library microservice hides the mode of operation from the users. The files can come from two sources:
The URL endpoint for this microservice is located at: `localhost:PORT/lib`

1. A directory on the server machine hosting DTaaS (`mode=local`)
1. A gitlab group on a gitlab instance (`mode=gitlab`). The gitlab can either be a community [gitlab](https://gitlab.com) or your own private gitlab instance.
13 changes: 13 additions & 0 deletions docs/admin/trial.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@

# Trial Installation

A single step install script is helpful in performing a trial run of the software. This script installs DTaaS software with default credentials and users on a Ubuntu server OS. You can use it to check a test installation of DTaaS.

```bash
wget https://github.com/INTO-CPS-Association/DTaaS/releases/download/v0.2.0/single-script-install.sh
bash single-script-install.sh
```

!!! warning

This test installation has default credentials and is thus highly insecure.
Loading

0 comments on commit f444898

Please sign in to comment.