Skip to content

Commit e211bae

Browse files
Improve developer experience when working with traderx and k8s (finos#232)
* Add optional builds * Update readme as per feedback
1 parent 6863fa3 commit e211bae

File tree

13 files changed

+213
-60
lines changed

13 files changed

+213
-60
lines changed

README.md

+118-37
Original file line numberDiff line numberDiff line change
@@ -50,11 +50,23 @@ The project consists of multiple moving parts, and you can see how things hang t
5050
| [trade-processor](trade-processor) | java/spring | Trade Feed consumer which processes trade/orders |
5151
| [web-front-end](web-front-end) | html/angular or react | Interactive UI for executing trades and viewing blotter. Note: the AngularJS GUI was an initial contribution and contains account management capabilities. The React GUI was contributed during a hack day and may not work for managing accounts, but it does work for executing trades and viewing the blotter |
5252

53-
## Installation
53+
## Check out Code
5454

5555
This is installed locally through normal git clone operations.
5656

57-
## Usage example (Manual)
57+
```
58+
git clone https://github.com/finos/traderX.git
59+
```
60+
61+
## Run locally
62+
63+
There are a number of ways to run TraderX locally. You should choose the method you are most comfortable with.
64+
- [Run locally manually](#run-locally-manually)
65+
- [Run locally within a Corporate Environments](#run-locally-within-a-corporate-environments)
66+
- [Run locally using Docker & Docker Compose](#run-locally-using-docker--docker-compose)
67+
- [Run locally using Kubernetes](#run-locally-using-kubernetes)
68+
69+
## Run locally manually
5870

5971
In order to get things working together, it is recommended to select a range of ports to provide all running processes with, so that the pieces can interconnect as needed. To run this all up 'by hand' here are default ports which are used, and you can easily export these variables to your favorite shell.
6072

@@ -87,31 +99,7 @@ trade-service
8799
web-front-end
88100
```
89101

90-
## Usage (Docker + Docker Compose)
91-
92-
The easiest way to run up the entire system is using Docker Compose. This should work on your local computer using Docker Desktop / Docker Compose (tested on Mac Silicon) and also in Github Codespaces.
93-
94-
### Codespaces
95-
If using Github Codespaces it is recommended you select an 8-core type machine with 32GB RAM to ensure all the components have the required resources to start.
96-
97-
To do this
98-
* Select the Green Code menu at the top of this page
99-
* Select the Codespace tab then click the three dots '...' and select 'New with options...'.
100-
* Change the machine type to '8-core' and click 'Create codespace'
101-
102-
As of writing, personal Github accounts receive 120 free core hours per month for using Codespaces, see the most recent details [here](https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces#monthly-included-storage-and-core-hours-for-personal-accounts)
103-
104-
Once you have cloned the repository locally or once your Codespace has started, from the root traderX directory run
105-
```
106-
docker compose up
107-
```
108-
On first run this will build all of the containers from the project specific Dockerfile's and then start them in the correct sequence.
109-
110-
The Docker containers are configured via Docker Compose to connect to a shred virtual network enabling them to communciate whether running on your local computer or via a Codespace.
111-
112-
Once everything has started the WebUI will be accessible at http://localhost:8080 (even if using a codespace, the localhost URL will be mapped through from your local browser to the Codespace).
113-
114-
## Local Building (Corporate Environments)
102+
## Run locally within a Corporate Environments
115103

116104
When building locally in your company, if you are using a corporate artifact repository, you might need to override certain settings such as mavenCentral() in gradle, for the Java projects.
117105

@@ -162,34 +150,127 @@ cd .corp
162150
./gradlew account-service:bootRun
163151
```
164152

165-
## Usage (K8s)
153+
## Run locally using Docker & Docker Compose
154+
155+
The easiest way to run up the entire system is using Docker Compose. This should work on your local computer using Docker Desktop / Docker Compose (tested on Mac Silicon) and also in Github Codespaces.
166156

167-
The following are instructions to build and deploy all TraderX apps to your local enviroment using [tilt](https://tilt.dev) and kustomize files.
157+
### Codespaces
158+
If using Github Codespaces it is recommended you select an 8-core type machine with 32GB RAM to ensure all the components have the required resources to start.
168159

169-
## Prerequistes
170-
- Running [Docker](https://www.docker.com/products/docker-desktop/) or similar
171-
- Running K8s - [Kind](https://kind.sigs.k8s.io/) /[Minikube](https://minikube.sigs.k8s.io/docs/start/)/[k3s](https://k3s.io/) or similar
172-
- Install an [Ingress Controller](https://kubernetes.github.io/ingress-nginx/deploy/)
160+
To do this
161+
* Select the Green Code menu at the top of this page
162+
* Select the Codespace tab then click the three dots '...' and select 'New with options...'.
163+
* Change the machine type to '8-core' and click 'Create codespace'
164+
165+
As of writing, personal Github accounts receive 120 free core hours per month for using Codespaces, see the most recent details [here](https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-github-codespaces#monthly-included-storage-and-core-hours-for-personal-accounts)
166+
167+
Once you have cloned the repository locally or once your Codespace has started, from the root traderX directory run
168+
```
169+
docker compose up
170+
```
171+
On first run this will build all of the containers from the project specific Dockerfile's and then start them in the correct sequence.
172+
173+
The Docker containers are configured via Docker Compose to connect to a shred virtual network enabling them to communciate whether running on your local computer or via a Codespace.
174+
175+
Once everything has started the WebUI will be accessible at http://localhost:8080 (even if using a codespace, the localhost URL will be mapped through from your local browser to the Codespace).
176+
177+
178+
## Run locally using Kubernetes
179+
180+
Another easy way to run up the entire system is using Kubernetes (K8s), which using [tilt.dev](https://tilt.dev) also allows you to easily swap in locally built images. The following are instructions to deploy all TraderX apps to your local enviroment using [tilt.dev](https://tilt.dev) and kustomize files.
181+
182+
### Prerequistes
183+
- Install and run [Docker](https://www.docker.com/products/docker-desktop/) or similar
184+
- Install and run K8s - this could be one of the following well know distributions
185+
- [K8s with Docker Desktop](https://docs.docker.com/desktop/kubernetes/)
186+
- [Kind](https://kind.sigs.k8s.io/)
187+
- [Minikube](https://minikube.sigs.k8s.io/docs/start/)
188+
- [k3s](https://k3s.io/) or similar
189+
- Install an [Ingress Controller](https://kubernetes.github.io/ingress-nginx/deploy/) - this will need to match your k8s distribution
190+
- Install [tilt.dev](https://tilt.dev)
173191

174192
### Preflight checks
175193

176194
```
177195
kubectl get pods
178196
```
179197

180-
or
198+
You should see the kube pods as well as your ingress controller. Example below:
181199

182-
Check your cluster is running using the epic tool `k9s` - https://k9scli.io/
200+
```
201+
NAMESPACE NAME READY STATUS RESTARTS AGE
202+
ingress-nginx ingress-nginx-admission-create-6rh79 0/1 Completed 0 38s
203+
ingress-nginx ingress-nginx-admission-patch-f4kdg 0/1 Completed 1 38s
204+
ingress-nginx ingress-nginx-controller-7d4db76476-7wfl2 1/1 Running 0 38s
205+
kube-system coredns-7db6d8ff4d-ntdcr 1/1 Running 0 4h8m
206+
kube-system coredns-7db6d8ff4d-ptfcs 1/1 Running 0 4h8m
207+
kube-system etcd-docker-desktop 1/1 Running 2 4h8m
208+
kube-system kube-apiserver-docker-desktop 1/1 Running 2 4h8m
209+
kube-system kube-controller-manager-docker-desktop 1/1 Running 2 4h8m
210+
kube-system kube-proxy-qt5z4 1/1 Running 0 4h8m
211+
kube-system kube-scheduler-docker-desktop 1/1 Running 10 4h8m
212+
kube-system storage-provisioner 1/1 Running 0 4h8m
213+
```
183214

184-
### Start Tilt
215+
### Start Tilt.dev
185216

186-
This command will build and start all locally built applications and deploy them to your local K8s environment.
217+
This command will deploy all the services to your local K8s environment.
218+
Note: Ensure all the [pre-requistes](#prerequistes) are complete
187219

220+
After cloning the repo, you should `cd` into the folder (`cd traderx`)
188221
```
222+
# cd into gitops repo
189223
cd ./gitops/local/
224+
225+
# Start Tilt
190226
tilt up
191227
```
192228

229+
Expected Console Output:
230+
```
231+
Tilt started on http://localhost:10350/
232+
v0.30.2, built 2022-06-06
233+
234+
(space) to open the browser
235+
(s) to stream logs (--stream=true)
236+
(t) to open legacy terminal mode (--legacy=true)
237+
(ctrl-c) to exit
238+
```
239+
240+
This will download all the images and start them in your local k8s cluster.
241+
242+
Launching the tilt.dev local webpage will allow you to monitor the progress by pressing the space bar. Or use one of the other options from the console.
243+
244+
Note: if you run `tilt up` and you get an error stating `template engine not found for: up.`. Your OS might be trying to run a ruby library, please check the tilt.dev installation instructions, or you path settings.
245+
246+
247+
### Start Tilt.dev
248+
249+
### Local Developement
250+
251+
With all the services running you can then chose which ones you actively build locally.
252+
253+
If you go to your local [Tiltfile](./gitops/local/Tiltfile) you simply need to uncomment all the lines, or just for the respective applicaiton that you want to work on. Those applications will be built locally and deployed to your cluster instead of the host images. For example. In your local [Tiltfile](./gitops/local/Tiltfile) uncommenting the line for position_service will ask that Tilt.dev builds the images and your locally built image is deployed in your local k8s cluster in place of the prebuilt image from the Github Container Registry.
254+
255+
```
256+
# Uncomment lines to use locally built version
257+
# docker_build('ghcr.io/finos/traderx/database', './../../database/.')
258+
# docker_build('ghcr.io/finos/traderx/account-service', './../../account-service/.')
259+
# docker_build('ghcr.io/finos/traderx/people-service', './../../people-service/.')
260+
docker_build('ghcr.io/finos/traderx/position-service', './../../position-service/.')
261+
# docker_build('ghcr.io/finos/traderx/reference-data', './../../reference-data/.')
262+
# docker_build('ghcr.io/finos/traderx/trade-feed', './../../trade-feed/.')
263+
# docker_build('ghcr.io/finos/traderx/trade-processor', './../../trade-processor/.')
264+
# docker_build('ghcr.io/finos/traderx/trade-service', './../../trade-service/.')
265+
# docker_build('ghcr.io/finos/traderx/web-front-end-angular', './../../web-front-end/angular/.')
266+
yaml = kustomize(('./traderx'))
267+
print(yaml)
268+
k8s_yaml(yaml)
269+
```
270+
271+
### Clean up
272+
273+
Simply stop tilt and then run `tilt down`
193274

194275
# Getting Involved
195276

gitops/base/account-service/deployment.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ spec:
2323
value: "18088"
2424
- name: DATABASE_TCP_HOST
2525
value: database
26-
image: traderx/account-service
26+
image: ghcr.io/finos/traderx/account-service
2727
imagePullPolicy: IfNotPresent
2828
ports:
2929
- containerPort: 18088

gitops/base/database/deployment.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ spec:
1616
spec:
1717
containers:
1818
- name: database-app
19-
image: traderx/database
19+
image: ghcr.io/finos/traderx/database
2020
imagePullPolicy: IfNotPresent
2121
ports:
2222
- containerPort: 18082

gitops/base/people-service/deployment.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ spec:
1818
spec:
1919
containers:
2020
- name: people-service
21-
image: traderx/people-service
21+
image: ghcr.io/finos/traderx/people-service
2222
imagePullPolicy: IfNotPresent
2323
ports:
2424
- containerPort: 18089

gitops/base/position-service/deployment.yaml

+23-1
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,33 @@ spec:
2323
value: database
2424
- name: POSITION_SERVICE_PORT
2525
value: "18090"
26-
image: traderx/position-service
26+
image: ghcr.io/finos/traderx/position-service
2727
imagePullPolicy: IfNotPresent
2828
ports:
2929
- containerPort: 18090
3030
protocol: TCP
3131
workingDir: /position-service
32+
33+
# Liveness Probe: Checks if the container is alive
34+
livenessProbe:
35+
httpGet:
36+
path: /health/alive # The endpoint for the liveness probe
37+
port: 18090 # Port to hit on the container
38+
initialDelaySeconds: 10 # Delay before starting the probe after the container starts
39+
periodSeconds: 10 # How often to run the probe (every 10 seconds)
40+
timeoutSeconds: 5 # Timeout for the probe to respond
41+
failureThreshold: 3 # If it fails 3 times, restart the container
42+
43+
# Readiness Probe: Checks if the container is ready to serve traffic
44+
readinessProbe:
45+
httpGet:
46+
path: /health/ready # The endpoint for the readiness probe
47+
port: 18090 # Port to hit on the container
48+
initialDelaySeconds: 5 # Delay before starting the probe after the container starts
49+
periodSeconds: 5 # How often to run the probe (every 5 seconds)
50+
timeoutSeconds: 5 # Timeout for the probe to respond
51+
successThreshold: 1 # Minimum number of successful probes to mark it as ready
52+
failureThreshold: 3 # If it fails 3 times, the pod is marked as unready
53+
3254
restartPolicy: Always
3355

gitops/base/reference-data/deployment.yaml

+2-2
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@ spec:
2020
- name: reference-data
2121
env:
2222
- name: REFERENCE_DATA_SERVICE_PORT
23-
value: "18095"
24-
image: traderx/reference-data
23+
value: "18085"
24+
image: ghcr.io/finos/traderx/reference-data
2525
imagePullPolicy: IfNotPresent
2626
ports:
2727
- containerPort: 18085

gitops/base/trade-feed/deployment.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ spec:
1717
app: trade-feed
1818
spec:
1919
containers:
20-
- image: traderx/trade-feed
20+
- image: ghcr.io/finos/traderx/trade-feed
2121
imagePullPolicy: IfNotPresent
2222
name: trade-feed
2323
env:

gitops/base/trade-processor/deployment.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ spec:
1616
spec:
1717
containers:
1818
- name: trade-processor-app
19-
image: traderx/trade-processor
19+
image: ghcr.io/finos/traderx/trade-processor
2020
imagePullPolicy: IfNotPresent
2121
ports:
2222
- containerPort: 18091

gitops/base/trade-service/deployment.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ spec:
2828
value: reference-data
2929
- name: TRADE_FEED_HOST
3030
value: trade-feed
31-
image: traderx/trade-service
31+
image: ghcr.io/finos/traderx/trade-service
3232
imagePullPolicy: IfNotPresent
3333
name: trade-service
3434
ports:

gitops/base/web-front-end/deployment.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ spec:
1717
app: web-front-end-angular
1818
spec:
1919
containers:
20-
- image: traderx/web-front-end-angular
20+
- image: ghcr.io/finos/traderx/web-front-end-angular
2121
imagePullPolicy: IfNotPresent
2222
name: web-front-end-angular
2323
env:

gitops/local/Tiltfile

+10-9
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,13 @@
1-
docker_build('traderx/database', './../../database/.')
2-
docker_build('traderx/account-service', './../../account-service/.')
3-
docker_build('traderx/people-service', './../../people-service/.')
4-
docker_build('traderx/position-service', './../../position-service/.')
5-
docker_build('traderx/reference-data', './../../reference-data/.')
6-
docker_build('traderx/trade-feed', './../../trade-feed/.')
7-
docker_build('traderx/trade-processor', './../../trade-processor/.')
8-
docker_build('traderx/trade-service', './../../trade-service/.')
9-
docker_build('traderx/web-front-end-angular', './../../web-front-end/angular/.')
1+
# Uncomment lines to use locally built version
2+
# docker_build('ghcr.io/finos/traderx/database', './../../database/.')
3+
# docker_build('ghcr.io/finos/traderx/account-service', './../../account-service/.')
4+
# docker_build('ghcr.io/finos/traderx/people-service', './../../people-service/.')
5+
# docker_build('ghcr.io/finos/traderx/position-service', './../../position-service/.')
6+
# docker_build('ghcr.io/finos/traderx/reference-data', './../../reference-data/.')
7+
# docker_build('ghcr.io/finos/traderx/trade-feed', './../../trade-feed/.')
8+
# docker_build('ghcr.io/finos/traderx/trade-processor', './../../trade-processor/.')
9+
# docker_build('ghcr.io/finos/traderx/trade-service', './../../trade-service/.')
10+
# docker_build('ghcr.io/finos/traderx/web-front-end-angular', './../../web-front-end/angular/.')
1011
yaml = kustomize(('./traderx'))
1112
print(yaml)
1213
k8s_yaml(yaml)

position-service/Dockerfile

+10-4
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,18 @@
22

33
# [Choice] Java version (use -bullseye variants on local arm64/Apple Silicon): 11, 17, 11-bullseye, 17-bullseye, 11-buster, 17-buster
44
ARG VARIANT="21"
5-
FROM mcr.microsoft.com/vscode/devcontainers/java:1-${VARIANT}
5+
FROM mcr.microsoft.com/vscode/devcontainers/java:1-${VARIANT} AS build-stage
66

7+
# Copy code
78
WORKDIR /position-service
89
COPY . .
910

10-
EXPOSE 18090
11-
11+
# Build code
1212
RUN ./gradlew build
13-
ENTRYPOINT ./gradlew bootRun
13+
14+
FROM mcr.microsoft.com/openjdk/jdk:${VARIANT}-ubuntu
15+
RUN mkdir /app
16+
COPY --from=build-stage /position-service/build/libs/*.jar /app/position-service.jar
17+
EXPOSE 18090
18+
# ENTRYPOINT ./gradlew bootRun
19+
ENTRYPOINT ["java","-jar","/app/position-service.jar"]
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
package finos.traderx.positionservice.controller;
2+
3+
import java.util.List;
4+
5+
import org.springframework.beans.factory.annotation.Autowired;
6+
import org.springframework.http.HttpStatus;
7+
import org.springframework.http.ResponseEntity;
8+
import org.springframework.web.bind.annotation.CrossOrigin;
9+
import org.springframework.web.bind.annotation.ExceptionHandler;
10+
import org.springframework.web.bind.annotation.GetMapping;
11+
import org.springframework.web.bind.annotation.PathVariable;
12+
import org.springframework.web.bind.annotation.RequestMapping;
13+
import org.springframework.web.bind.annotation.RestController;
14+
15+
import finos.traderx.positionservice.model.Position;
16+
import finos.traderx.positionservice.service.PositionService;
17+
18+
@CrossOrigin("*")
19+
@RestController
20+
@RequestMapping(value="/health", produces="application/json")
21+
public class HealthController {
22+
23+
@Autowired
24+
PositionService positionService;
25+
26+
27+
@GetMapping("/ready")
28+
public ResponseEntity isReady() {
29+
List<Position> retVal = this.positionService.getAllPositions();
30+
return ResponseEntity.ok(retVal.size()>0);
31+
}
32+
33+
@GetMapping("/alive")
34+
public ResponseEntity<Boolean> isAlive() {
35+
return ResponseEntity.ok(true);
36+
}
37+
38+
39+
@ExceptionHandler(Exception.class)
40+
public ResponseEntity<String> generalError(Exception e) {
41+
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(e.getMessage());
42+
}
43+
}

0 commit comments

Comments
 (0)