POST /apis/v1/applications/operations
Request +
curl --location 'http://drove.local:7000/apis/v1/operations' \
+--header 'Content-Type: application/json' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+--data '{
+ "type": "SCALE",
+ "appId": "TEST_APP-1",
+ "requiredInstances": 1,
+ "opSpec": {
+ "timeout": "1m",
+ "parallelism": 20,
+ "failureStrategy": "STOP"
+ }
+}'
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "appId": "TEST_APP-1"
+ },
+ "message": "success"
+}
+Tip
+Relevant payloads for application commands can be found in application operations section.
+POST /apis/v1/applications/operations/{appId}/cancel
Request +
curl --location --request POST 'http://drove.local:7000/apis/v1/operations/TEST_APP/cancel' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+--data ''
+Response +
{
+ "status": "SUCCESS",
+ "message": "success"
+}
+GET /apis/v1/applications
Request +
curl --location 'http://drove.local:7000/apis/v1/applications' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "TEST_APP-1": {
+ "id": "TEST_APP-1",
+ "name": "TEST_APP",
+ "requiredInstances": 0,
+ "healthyInstances": 0,
+ "totalCPUs": 0,
+ "totalMemory": 0,
+ "state": "MONITORING",
+ "created": 1719826995764,
+ "updated": 1719892126096
+ }
+ },
+ "message": "success"
+}
+GET /apis/v1/applications/{id}
Request +
curl --location 'http://drove.local:7000/apis/v1/applications/TEST_APP-1' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "id": "TEST_APP-1",
+ "name": "TEST_APP",
+ "requiredInstances": 1,
+ "healthyInstances": 1,
+ "totalCPUs": 1,
+ "totalMemory": 128,
+ "state": "RUNNING",
+ "created": 1719826995764,
+ "updated": 1719892279019
+ },
+ "message": "success"
+}
+GET /apis/v1/applications/{id}/spec
Request
+curl --location 'http://drove.local:7000/apis/v1/applications/TEST_APP-1/spec' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "name": "TEST_APP",
+ "version": "1",
+ "executable": {
+ "type": "DOCKER",
+ "url": "ghcr.io/appform-io/perf-test-server-httplib",
+ "dockerPullTimeout": "100 seconds"
+ },
+ "exposedPorts": [
+ {
+ "name": "main",
+ "port": 8000,
+ "type": "HTTP"
+ }
+ ],
+ "volumes": [],
+ "configs": [
+ {
+ "type": "INLINE",
+ "localFilename": "/testfiles/drove.txt",
+ "data": ""
+ }
+ ],
+ "type": "SERVICE",
+ "resources": [
+ {
+ "type": "CPU",
+ "count": 1
+ },
+ {
+ "type": "MEMORY",
+ "sizeInMB": 128
+ }
+ ],
+ "placementPolicy": {
+ "type": "ANY"
+ },
+ "healthcheck": {
+ "mode": {
+ "type": "HTTP",
+ "protocol": "HTTP",
+ "portName": "main",
+ "path": "/",
+ "verb": "GET",
+ "successCodes": [
+ 200
+ ],
+ "payload": "",
+ "connectionTimeout": "1 second",
+ "insecure": false
+ },
+ "timeout": "1 second",
+ "interval": "5 seconds",
+ "attempts": 3,
+ "initialDelay": "0 seconds"
+ },
+ "readiness": {
+ "mode": {
+ "type": "HTTP",
+ "protocol": "HTTP",
+ "portName": "main",
+ "path": "/",
+ "verb": "GET",
+ "successCodes": [
+ 200
+ ],
+ "payload": "",
+ "connectionTimeout": "1 second",
+ "insecure": false
+ },
+ "timeout": "1 second",
+ "interval": "3 seconds",
+ "attempts": 3,
+ "initialDelay": "0 seconds"
+ },
+ "tags": {
+ "superSpecialApp": "yes_i_am",
+ "say_my_name": "heisenberg"
+ },
+ "env": {
+ "CORES": "8"
+ },
+ "exposureSpec": {
+ "vhost": "testapp.local",
+ "portName": "main",
+ "mode": "ALL"
+ },
+ "preShutdown": {
+ "hooks": [
+ {
+ "type": "HTTP",
+ "protocol": "HTTP",
+ "portName": "main",
+ "path": "/",
+ "verb": "GET",
+ "successCodes": [
+ 200
+ ],
+ "payload": "",
+ "connectionTimeout": "1 second",
+ "insecure": false
+ }
+ ],
+ "waitBeforeKill": "3 seconds"
+ }
+ },
+ "message": "success"
+}
+Note
+configs section data will not be returned by any api calls
GET /apis/v1/applications/{id}/instances
Request +
curl --location 'http://drove.local:7000/apis/v1/applications/TEST_APP-1/instances' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": [
+ {
+ "appId": "TEST_APP-1",
+ "appName": "TEST_APP",
+ "instanceId": "AI-58eb1111-8c2c-4ea2-a159-8fc68010a146",
+ "executorId": "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d",
+ "localInfo": {
+ "hostname": "ppessdev",
+ "ports": {
+ "main": {
+ "containerPort": 8000,
+ "hostPort": 33857,
+ "portType": "HTTP"
+ }
+ }
+ },
+ "resources": [
+ {
+ "type": "CPU",
+ "cores": {
+ "0": [
+ 2
+ ]
+ }
+ },
+ {
+ "type": "MEMORY",
+ "memoryInMB": {
+ "0": 128
+ }
+ }
+ ],
+ "state": "HEALTHY",
+ "metadata": {},
+ "errorMessage": "",
+ "created": 1719892354194,
+ "updated": 1719893180105
+ }
+ ],
+ "message": "success"
+}
+GET /apis/v1/applications/{id}/instances/old
Request +
curl --location 'http://drove.local:7000/apis/v1/applications/TEST_APP-1/instances/old' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": [
+ {
+ "appId": "TEST_APP-1",
+ "appName": "TEST_APP",
+ "instanceId": "AI-869e34ed-ebf3-4908-bf48-719475ca5640",
+ "executorId": "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d",
+ "resources": [
+ {
+ "type": "CPU",
+ "cores": {
+ "0": [
+ 2
+ ]
+ }
+ },
+ {
+ "type": "MEMORY",
+ "memoryInMB": {
+ "0": 128
+ }
+ }
+ ],
+ "state": "STOPPED",
+ "metadata": {},
+ "errorMessage": "Error while pulling image ghcr.io/appform-io/perf-test-server-httplib: Status 500: {\"message\":\"Get \\\"https://ghcr.io/v2/\\\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)\"}\n",
+ "created": 1719892279039,
+ "updated": 1719892354099
+ }
+ ],
+ "message": "success"
+}
+GET /apis/v1/applications/{appId}/instances/{instanceId}
Request +
curl --location 'http://drove.local:7000/apis/v1/applications/TEST_APP-1/instances/AI-58eb1111-8c2c-4ea2-a159-8fc68010a146' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "appId": "TEST_APP-1",
+ "appName": "TEST_APP",
+ "instanceId": "AI-58eb1111-8c2c-4ea2-a159-8fc68010a146",
+ "executorId": "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d",
+ "localInfo": {
+ "hostname": "ppessdev",
+ "ports": {
+ "main": {
+ "containerPort": 8000,
+ "hostPort": 33857,
+ "portType": "HTTP"
+ }
+ }
+ },
+ "resources": [
+ {
+ "type": "CPU",
+ "cores": {
+ "0": [
+ 2
+ ]
+ }
+ },
+ {
+ "type": "MEMORY",
+ "memoryInMB": {
+ "0": 128
+ }
+ }
+ ],
+ "state": "HEALTHY",
+ "metadata": {},
+ "errorMessage": "",
+ "created": 1719892354194,
+ "updated": 1719893440105
+ },
+ "message": "success"
+}
+GET /apis/v1/endpoints
Info
+This API provides up-to-date information about the host and port information about application instances running on the cluster. This information can be used for Service Discovery systems to keep their information in sync with changes in the topology of applications running on the cluster.
+Tip
+Any tag specified in the application specification is also exposed on endpoint. This can be used to implement complicated routing logic if needed in the NGinx template on Drove Gateway.
Request +
curl --location 'http://drove.local:7000/apis/v1/endpoints' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": [
+ {
+ "appId": "TEST_APP-1",
+ "vhost": "testapp.local",
+ "tags": {
+ "superSpecialApp": "yes_i_am",
+ "say_my_name": "heisenberg"
+ },
+ "hosts": [
+ {
+ "host": "ppessdev",
+ "port": 44315,
+ "portType": "HTTP"
+ }
+ ]
+ },
+ {
+ "appId": "TEST_APP-2",
+ "vhost": "testapp.local",
+ "tags": {
+ "superSpecialApp": "yes_i_am",
+ "say_my_name": "heisenberg"
+ },
+ "hosts": [
+ {
+ "host": "ppessdev",
+ "port": 46623,
+ "portType": "HTTP"
+ }
+ ]
+ }
+ ],
+ "message": "success"
+}
+GET /apis/v1/ping
Request +
curl --location 'http://drove.local:7000/apis/v1/ping' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": "pong",
+ "message": "success"
+}
+Tip
+Use this api call to determine the leader in a cluster. This api will return a HTTP 200 only for the leader controller. All other controllers in the cluster will return 4xx for this api call.
+GET /apis/v1/cluster
Request +
curl --location 'http://drove.local:7000/apis/v1/cluster' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "leader": "ppessdev:4000",
+ "state": "NORMAL",
+ "numExecutors": 1,
+ "numApplications": 1,
+ "numActiveApplications": 1,
+ "freeCores": 9,
+ "usedCores": 1,
+ "totalCores": 10,
+ "freeMemory": 18898,
+ "usedMemory": 128,
+ "totalMemory": 19026
+ },
+ "message": "success"
+}
+POST /apis/v1/cluster/maintenance/set
Request +
curl --location --request POST 'http://drove.local:7000/apis/v1/cluster/maintenance/set' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+--data ''
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "state": "MAINTENANCE",
+ "updated": 1719897526772
+ },
+ "message": "success"
+}
+POST /apis/v1/cluster/maintenance/unset
Request +
curl --location --request POST 'http://drove.local:7000/apis/v1/cluster/maintenance/unset' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+--data ''
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "state": "NORMAL",
+ "updated": 1719897573226
+ },
+ "message": "success"
+}
+Warning
+Cluster will remain in maintenance mode for some time (about 2 minutes) internally even after maintenance mode is removed.
+GET /apis/v1/cluster/executors
Request +
curl --location 'http://drove.local:7000/apis/v1/cluster/executors' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": [
+ {
+ "executorId": "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d",
+ "hostname": "ppessdev",
+ "port": 3000,
+ "transportType": "HTTP",
+ "freeCores": 9,
+ "usedCores": 1,
+ "freeMemory": 18898,
+ "usedMemory": 128,
+ "tags": [
+ "ppessdev"
+ ],
+ "state": "ACTIVE"
+ }
+ ],
+ "message": "success"
+}
+GET /apis/v1/cluster/executors/{id}
Request +
curl --location 'http://drove.local:7000/apis/v1/cluster/executors/a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+{
+ "status": "SUCCESS",
+ "data": {
+ "type": "EXECUTOR",
+ "hostname": "ppessdev",
+ "port": 3000,
+ "transportType": "HTTP",
+ "updated": 1719897100104,
+ "state": {
+ "executorId": "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d",
+ "cpus": {
+ "type": "CPU",
+ "freeCores": {
+ "0": [
+ 3,
+ 4,
+ 5,
+ 6,
+ 7,
+ 8,
+ 9,
+ 10,
+ 11
+ ]
+ },
+ "usedCores": {
+ "0": [
+ 2
+ ]
+ }
+ },
+ "memory": {
+ "type": "MEMORY",
+ "freeMemory": {
+ "0": 18898
+ },
+ "usedMemory": {
+ "0": 128
+ }
+ }
+ },
+ "instances": [
+ {
+ "appId": "TEST_APP-1",
+ "appName": "TEST_APP",
+ "instanceId": "AI-58eb1111-8c2c-4ea2-a159-8fc68010a146",
+ "executorId": "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d",
+ "localInfo": {
+ "hostname": "ppessdev",
+ "ports": {
+ "main": {
+ "containerPort": 8000,
+ "hostPort": 33857,
+ "portType": "HTTP"
+ }
+ }
+ },
+ "resources": [
+ {
+ "type": "CPU",
+ "cores": {
+ "0": [
+ 2
+ ]
+ }
+ },
+ {
+ "type": "MEMORY",
+ "memoryInMB": {
+ "0": 128
+ }
+ }
+ ],
+ "state": "HEALTHY",
+ "metadata": {},
+ "errorMessage": "",
+ "created": 1719892354194,
+ "updated": 1719897100104
+ }
+ ],
+ "tasks": [],
+ "tags": [
+ "ppessdev"
+ ],
+ "blacklisted": false
+ },
+ "message": "success"
+}
+POST /apis/v1/cluster/executors/blacklist
Request +
curl --location --request POST 'http://drove.local:7000/apis/v1/cluster/executors/blacklist?id=a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+--data ''
+Note
+Unlike other POST apis, the executors to be blacklisted are passed as query parameter id. To blacklist multiple executors, pass .../blacklist?id=<id1>&id=<id2>...
Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "successful": [
+ "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d"
+ ],
+ "failed": []
+ },
+ "message": "success"
+}
+POST /apis/v1/cluster/executors/unblacklist
Request +
curl --location --request POST 'http://drove.local:7000/apis/v1/cluster/executors/unblacklist?id=a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+--data ''
+Note
+Unlike other POST apis, the executors to be un-blacklisted are passed as query parameter id. To un-blacklist multiple executors, pass .../unblacklist?id=<id1>&id=<id2>...
Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "successful": [
+ "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d"
+ ],
+ "failed": []
+ },
+ "message": "success"
+}
+The following APIs can be used to monitor events on Drove. If the data needs to be consumed, the /latest API should be used. For simply knowing if an event of a certain type has occurred or not, the /summary is sufficient.
GET /apis/v1/cluster/events/latest
Request +
curl --location 'http://drove.local:7000/apis/v1/cluster/events/latest?size=1024&lastSyncTime=0' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "events": [
+ {
+ "metadata": {
+ "CURRENT_INSTANCES": 0,
+ "APP_ID": "TEST_APP-1",
+ "PLACEMENT_POLICY": "ANY",
+ "APP_VERSION": "1",
+ "CPU_COUNT": 1,
+ "CURRENT_STATE": "RUNNING",
+ "PORTS": "main:8000:http",
+ "MEMORY": 128,
+ "EXECUTABLE": "ghcr.io/appform-io/perf-test-server-httplib",
+ "VHOST": "testapp.local",
+ "APP_NAME": "TEST_APP"
+ },
+ "type": "APP_STATE_CHANGE",
+ "id": "a2b7d673-2bc2-4084-8415-d8d37cafa63d",
+ "time": 1719977632050
+ },
+ {
+ "metadata": {
+ "APP_NAME": "TEST_APP",
+ "APP_ID": "TEST_APP-1",
+ "PORTS": "main:44315:http",
+ "EXECUTOR_ID": "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d",
+ "EXECUTOR_HOST": "ppessdev",
+ "CREATED": 1719977629042,
+ "INSTANCE_ID": "AI-5efbb94f-835c-4c62-a073-a68437e60339",
+ "CURRENT_STATE": "HEALTHY"
+ },
+ "type": "INSTANCE_STATE_CHANGE",
+ "id": "55d5876f-94ac-4c5d-a580-9c3b296add46",
+ "time": 1719977631534
+ }
+ ],
+ "lastSyncTime": 1719977632050//(1)!
+ },
+ "message": "success"
+}
+lastSyncTime in the next call to events api to receive latest events.| Query Parameter | +Validation | +Description | +
|---|---|---|
| lastSyncTime | ++ve long range | +Time when the last sync call happened on the server. Defaults to 0 (initial sync). | +
| size | +1-1024 | +Number of latest events to return. Defaults to 1024. We recommend leaving this as is. | +
GET /apis/v1/cluster/events/summary
Request +
curl --location 'http://drove.local:7000/apis/v1/cluster/events/summary?lastSyncTime=0' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+{
+ "status": "SUCCESS",
+ "data": {
+ "eventsCount": {
+ "INSTANCE_STATE_CHANGE": 8,
+ "APP_STATE_CHANGE": 17,
+ "EXECUTOR_BLACKLISTED": 1,
+ "EXECUTOR_UN_BLACKLISTED": 1
+ },
+ "lastSyncTime": 1719977632050//(1)!
+ },
+ "message": "success"
+}
+lastSyncTime in the next call to events api to receive latest events.This is applicable for both the APIs listed above
+lastSyncTime as zero.lastSyncTimelastSyncTime as the lastSyncTime param in the next callInfo
+Model for the events can be found here.
+Tip
+Java programs should definitely look at using the event listener library +to listen to cluster events
+This section lists all the APIs that a user can communicate with.
+Use a standard HTTP client in the language of your choice to make a call to the leader controller (the cluster virtual host exposed by drove-gateway-nginx).
+Tip
+In case you are using Java, we recommend using the drove-client library along with the http-transport.
+++If multiple controllers endpoints are provided, the client will track the leader automatically. This will reduce your dependency on drove-gateway.
+
Drove uses basic auth for authentication. (You can extend to use any other auth format like OAuth). The basic auth credentials need to be sent out in the standard format in the Authorization header.
The response format is standard for all API calls:
+{
+ "status": "SUCCESS",//(1)!
+ "data": {//(2)!
+ "taskId": "T0012"
+ },
+ "message": "success"//(3)!
+}
+SUCCESS or FAILURE as the case may be.success if the call was successful or relevant error message.Warning
+APIs will return relevant HTTP status codes in case of error (for example 400 for validation errors, 401 for authentication failure). However, you must always ensure that the status field is set to SUCCESS for assuming the api call is succesful, even when HTTP status code is 2xx.
APIs in Drove belong to the following major classes:
+ +Tip
+Response models for these apis can be found in drove-models
+Note
+There are no publicly accessible APIs exposed by individual executors.
+Application
+GET /apis/v1/logfiles/applications/{appId}/{instanceId}/list
Task
+GET /apis/v1/logfiles/tasks/{sourceAppName}/{taskId}/list
Request +
curl --location 'http://drove.local:7000/apis/v1/logfiles/applications/TEST_APP-1/AI-5efbb94f-835c-4c62-a073-a68437e60339/list' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "files": [
+ "output.log-2024-07-04",
+ "output.log-2024-07-03",
+ "output.log"
+ ]
+}
+Application
+GET /apis/v1/logfiles/applications/{appId}/{instanceId}/download/{fileName}
Task
+GET /apis/v1/logfiles/tasks/{sourceAppName}/{taskId}/download/{fileName}
Request +
curl --location 'http://drove.local:7000/apis/v1/logfiles/applications/TEST_APP-1/AI-5efbb94f-835c-4c62-a073-a68437e60339/download/output.log' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response
+
Note
+The Content-Disposition header is set properly to the actual filename. For the above example it would be set to attachment; filename=output.log.
Application
+GET /apis/v1/logfiles/applications/{appId}/{instanceId}/read/{fileName}
Task
+GET /apis/v1/logfiles/tasks/{sourceAppName}/{taskId}/read/{fileName}
| Query Parameter | +Validation | +Description | +
|---|---|---|
| offset | +Default -1, should be positive number | +The offset of the file to read from. | +
| length | +Should be a positive number | +Number of bytes to read. | +
Request +
curl --location 'http://drove.local:7000/apis/v1/logfiles/applications/TEST_APP-1/AI-5efbb94f-835c-4c62-a073-a68437e60339/read/output.log' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "data": "", //(1)!
+ "offset": 43318 //(2)!
+}
+/read api with offset=-1, length = buffer sizedata returned might be empty or less than length depending on availability.Warning
+tail type functionalityPOST /apis/v1/tasks/operations
Request +
curl --location 'http://drove.local:7000/apis/v1/tasks/operations' \
+--header 'Content-Type: application/json' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+--data '{
+ "type": "KILL",
+ "sourceAppName" : "TEST_APP",
+ "taskId" : "T0012",
+ "opSpec": {
+ "timeout": "5m",
+ "parallelism": 1,
+ "failureStrategy": "STOP"
+ }
+}'
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "taskId": "T0012"
+ },
+ "message": "success"
+}
+Tip
+Relevant payloads for task commands can be found in task operations section.
+POST /apis/v1/tasks/search
GET /apis/v1/tasks
Request +
curl --location 'http://drove.local:7000/apis/v1/tasks' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": [
+ {
+ "sourceAppName": "TEST_APP",
+ "taskId": "T0013",
+ "instanceId": "TI-c2140806-2bb5-4ed3-9bb9-0c0c5fd0d8d6",
+ "executorId": "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d",
+ "hostname": "ppessdev",
+ "executable": {
+ "type": "DOCKER",
+ "url": "ghcr.io/appform-io/test-task",
+ "dockerPullTimeout": "100 seconds"
+ },
+ "resources": [
+ {
+ "type": "CPU",
+ "cores": {
+ "0": [
+ 2
+ ]
+ }
+ },
+ {
+ "type": "MEMORY",
+ "memoryInMB": {
+ "0": 512
+ }
+ }
+ ],
+ "volumes": [],
+ "env": {
+ "ITERATIONS": "10"
+ },
+ "state": "RUNNING",
+ "metadata": {},
+ "errorMessage": "",
+ "created": 1719827035480,
+ "updated": 1719827038414
+ }
+ ],
+ "message": "success"
+}
+GET /apis/v1/tasks/{sourceAppName}/instances/{taskId}
Request +
curl --location 'http://drove.local:7000/apis/v1/tasks/TEST_APP/instances/T0012' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4='
+Response +
{
+ "status": "SUCCESS",
+ "data": {
+ "sourceAppName": "TEST_APP",
+ "taskId": "T0012",
+ "instanceId": "TI-6cf36f5c-6480-4ed5-9e2d-f79d9648529a",
+ "executorId": "a45442a1-d4d0-3479-ab9e-3ed0aa5f7d2d",
+ "hostname": "ppessdev",
+ "executable": {
+ "type": "DOCKER",
+ "url": "ghcr.io/appform-io/test-task",
+ "dockerPullTimeout": "100 seconds"
+ },
+ "resources": [
+ {
+ "type": "CPU",
+ "cores": {
+ "0": [
+ 3
+ ]
+ }
+ },
+ {
+ "type": "MEMORY",
+ "memoryInMB": {
+ "0": 512
+ }
+ }
+ ],
+ "volumes": [],
+ "env": {
+ "ITERATIONS": "10"
+ },
+ "state": "STOPPED",
+ "metadata": {},
+ "taskResult": {
+ "status": "SUCCESSFUL",
+ "exitCode": 0
+ },
+ "errorMessage": "",
+ "created": 1719823470267,
+ "updated": 1719823483836
+ },
+ "message": "success"
+}
+An application is a virtual representation of a running service in the system.
+Running containers for an application are called application instances.
+An application specification contains the following details about the application:
+Info
+Once a spec is registered to the cluster, it can not be changed
+Once an application is created on the cluster, an Application id is generated. The format of this id currently is: {name}-{version}. All further operations to be done on the application will need to refer to it by this ID.
An application on a Drove cluster follows a fixed lifecycle modelled as a state machine. State transitions are triggered by operations. Operations can be issued externally using API calls or may be generated internally by the application monitoring system.
+Applications on a Drove cluster can be one of the following states:
+The following application operations are recognized by Drove:
+Tip
+All operations can take an optional Cluster Operation Spec which can be used to control the timeout and parallelism of tasks generated by the operation.
+The following state machine signifies the states and transitions as affected by cluster state and operations issued.
+
Application instances are running containers for an application. The state machine for instances are managed in a decentralised manner on the cluster nodes locally and not by the controllers. This includes running health checks, readiness checks and shutdown hooks on the container, container loss detection and container state recovery on executor service restart.
+Regular updates about the instance state are provided by executors to the controllers and are used to keep the application state up-to-date or trigger application operations to bring the applications to stable states.
+An application instance can be in one of the following states at one point in time:
+Instance state machine transitions might be triggered on receipt of commands issued by the controller or due to internal changes in the container (might have died or started failing health checks) as well as external factors like executor service restarts.
+
Note
+No operations are allowed to be performed on application instances directly through the executor
+This page discusses operations relevant to Application management. Please go over the Application State Machine and Application Instance State Machine to understand the different states an application (and it's instances) can be in and how operations applied move an application from one state to another.
+Note
+Please go through Cluster Op Spec to understand the operation parameters being sent.
+Note
+Only one operation can be active on a particular {appName,version} combination.
Warning
+Only the leader controller will accept and process operations. To avoid confusion, use the controller endpoint exposed by Drove Gateway to issue commands.
+Tip
+Use the Drove CLI to perform all manual operations.
+All operations for application lifecycle management need to be issued via a POST HTTP call to the leader controller endpoint on the path /apis/v1/applications/operations. API will return HTTP OK/200 and relevant json response as payload.
Sample api call:
+curl --location 'http://drove.local:7000/apis/v1/applications/operations' \
+--header 'Content-Type: application/json' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+--data '{
+ "type": "START_INSTANCES",
+ "appId": "TEST_APP-3",
+ "instances": 1,
+ "opSpec": {
+ "timeout": "5m",
+ "parallelism": 32,
+ "failureStrategy": "STOP"
+ }
+}'
+Note
+In the above examples, http://drove.local:7000 is the endpoint of the leader. TEST_APP-3 is the Application ID. Authorization is basic auth.
When an operation is submitted to the cluster, a cluster op spec needs to be specified. This is needed to control different aspects of the operation, including parallelism of an operation or increase the timeout for the operation and so on.
+The following aspects of an operation can be configured:
+| Name | +Option | +Description | +
|---|---|---|
| Timeout | +timeout |
+The duration after which Drove considers the operation to have timed out. | +
| Parallelism | +parallelism |
+Parallelism of the task. (Range: 1-32) | +
| Failure Strategy | +failureStrategy |
+Set this to STOP. |
+
Note
+For internal recovery operations, Drove generates it's own operations. For that, Drove applies the following cluster operation spec:
+STOP++The default operation spec can be configured in the controller configuration file. It is recommended to set this to a something like 8 for faster recovery.
+
Operations can be requested to be cancelled asynchronously. A POST call needs to be made to leader controller endpoint on the api /apis/v1/operations/{applicationId}/cancel (1) to achieve this.
applicationId is the Application ID for the applicationcurl --location --request POST 'http://drove.local:7000/apis/v1/operations/TEST_APP-3/cancel' \
+--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
+--data ''
+Warning
+Operation cancellation is not instantaneous. Cancellation will be affected only after current execution of the active operation is complete.
+Before deploying containers on the cluster, an application needs to be created.
+Preconditions:
+State Transition:
+MONITORINGTo create an application, an Application Spec needs to be created first.
+Once ready, CLI command needs to be issued or the following payload needs to be sent:
+drove -c local apps create sample/test_app.json
+Sample Request Payload +
{
+ "type": "CREATE",
+ "spec": {...}, //(1)!
+ "opSpec": { //(2)!
+ "timeout": "5m",
+ "parallelism": 1,
+ "failureStrategy": "STOP"
+ }
+}
+Sample response +
{
+ "data" : {
+ "appId" : "TEST_APP-1"
+ },
+ "message" : "success",
+ "status" : "SUCCESS"
+}
+New instances can be started by issuing the START_INSTANCES command.
Preconditions
+- Application must be in one of the following states: MONITORING, RUNNING
State Transition:
+RUNNING, MONITORING} → RUNNINGThe following command/payload will start 2 new instances of the application.
drove -c local apps deploy TEST_APP-1 2
+Sample Request Payload +
{
+ "type": "START_INSTANCES",
+ "appId": "TEST_APP-1",//(1)!
+ "instances": 2,//(2)!
+ "opSpec": {//(3)!
+ "timeout": "5m",
+ "parallelism": 32,
+ "failureStrategy": "STOP"
+ }
+}
+Sample response +
{
+ "status": "SUCCESS",
+ "data": {
+ "appId": "TEST_APP-1"
+ },
+ "message": "success"
+}
+All instances of an application can be shut down by issuing the SUSPEND command.
Preconditions
+- Application must be in one of the following states: MONITORING, RUNNING
State Transition:
+RUNNING, MONITORING} → MONITORINGThe following command/payload will suspend all instances of the application.
+drove -c local apps suspend TEST_APP-1
+Sample Request Payload +
{
+ "type": "SUSPEND",
+ "appId": "TEST_APP-1",//(1)!
+ "opSpec": {//(2)!
+ "timeout": "5m",
+ "parallelism": 32,
+ "failureStrategy": "STOP"
+ }
+}
+Sample response +
{
+ "status": "SUCCESS",
+ "data": {
+ "appId": "TEST_APP-1"
+ },
+ "message": "success"
+}
+Scaling the application to required number of containers can be achieved using the SCALE command. Application can be either scaled up or down using this command.
Preconditions
+- Application must be in one of the following states: MONITORING, RUNNING
State Transition:
+RUNNING, MONITORING} → MONITORING if requiredInstances is set to 0RUNNING, MONITORING} → RUNNING if requiredInstances is non 0drove -c local apps scale TEST_APP-1 2
+Sample Request Payload +
{
+ "type": "SCALE",
+ "appId": "TEST_APP-1", //(3)!
+ "requiredInstances": 2, //(1)!
+ "opSpec": { //(2)!
+ "timeout": "1m",
+ "parallelism": 20,
+ "failureStrategy": "STOP"
+ }
+}
+Sample response +
{
+ "status": "SUCCESS",
+ "data": {
+ "appId": "TEST_APP-1"
+ },
+ "message": "success"
+}
+Note
+During scale down, older instances are stopped first
+Tip
+If implementing automation on top of Drove APIs, just use the SCALE command to scale up or down instead of using START_INSTANCES or SUSPEND separately.
Application can be restarted by issuing the REPLACE_INSTANCES operation. In this case, first clusterOpSpec.parallelism number of containers are spun up first and then an equivalent number of them are spun down. This ensures that cluster maintains enough capacity is maintained in the cluster to handle incoming traffic as the restart is underway.
Warning
+If the cluster does not have sufficient capacity to spin up new containers, this operation will get stuck. So adjust your parallelism accordingly.
+Preconditions
+- Application must be in RUNNING state.
State Transition:
+RUNNING → REPLACE_INSTANCES_REQUESTED → RUNNINGdrove -c local apps restart TEST_APP-1
+Sample Request Payload +
{
+ "type": "REPLACE_INSTANCES",
+ "appId": "TEST_APP-1", //(1)!
+ "instanceIds": [], //(2)!
+ "opSpec": { //(3)!
+ "timeout": "1m",
+ "parallelism": 20,
+ "failureStrategy": "STOP"
+ }
+}
+Sample response +
{
+ "status": "SUCCESS",
+ "data": {
+ "appId": "TEST_APP-1"
+ },
+ "message": "success"
+}
+Tip
+To replace specific instances, pass their application instance ids (starts with AI-...) in the instanceIds parameter in the JSON payload.
Application instances can be killed by issuing the STOP_INSTANCES operation. Default behaviour of Drove is to replace killed instances by new instances. Such new instances are always spun up before the specified(old) instances are stopped. If skipRespawn parameter is set to true, the application instance is killed but no new instances are spun up to replace it.
Warning
+If the cluster does not have sufficient capacity to spin up new containers, and skipRespawn is not set or set to false, this operation will get stuck.
Preconditions
+- Application must be in RUNNING state.
State Transition:
+RUNNING → STOP_INSTANCES_REQUESTED → RUNNING if final number of instances is non zeroRUNNING → STOP_INSTANCES_REQUESTED → MONITORING if final number of instances is zerodrove -c local apps appinstances kill TEST_APP-1 AI-601d160e-c692-4ddd-8b7f-4c09b30ed02e
+Sample Request Payload +
{
+ "type": "STOP_INSTANCES",
+ "appId" : "TEST_APP-1",//(1)!
+ "instanceIds" : [ "AI-601d160e-c692-4ddd-8b7f-4c09b30ed02e" ],//(2)!
+ "skipRespawn" : true,//(3)!
+ "opSpec": {//(4)!
+ "timeout": "5m",
+ "parallelism": 1,
+ "failureStrategy": "STOP"
+ }
+}
+false by default.Sample response +
{
+ "status": "SUCCESS",
+ "data": {
+ "appId": "TEST_APP-1"
+ },
+ "message": "success"
+}
+To remove an application deployment (appName-version combo) the DESTROY command can be issued.
Preconditions:
+State Transition:
+MONITORING → DESTROY_REQUESTED → DESTROYED → noneTo create an application, an Application Spec needs to be created first.
+Once ready, CLI command needs to be issued or the following payload needs to be sent:
+drove -c local apps destroy TEST_APP_1
+Sample Request Payload +
{
+ "type": "DESTROY",
+ "appId" : "TEST_APP-1",//(1)!
+ "opSpec": {//(2)!
+ "timeout": "5m",
+ "parallelism": 2,
+ "failureStrategy": "STOP"
+ }
+}
+Sample response +
{
+ "status": "SUCCESS",
+ "data": {
+ "appId": "TEST_APP-1"
+ },
+ "message": "success"
+}
+Warning
+All metadata for an app and it's instances are completely obliterated from Drove's storage once an app is destroyed
+Drove tracks all instances for an app deployment in the cluster. It will ensure the required number of containers is always running on the cluster.
+Executor runs periodic health checks on the container according to check spec configuration. +- Runs readiness checks to ensure container is started properly before declaring it healthy +- Runs health checks on the container at regular intervals to ensure it is in operating condition
+Behavior for both is configured by setting the appropriate options in the application specification.
+Result of such health checks (both success and failure) are reported to the controller. Appropriate action is taken to shut down containers that fail readiness or health checks.
+If container for an application crashes, Drove will automatically spin up a container in it's place.
+If an executor node fails, instances running on that node will be lost. This is detected by the outage detector and new containers are spun up on other parts of the cluster.
+On restart, executor service reads the metadata embedded in the container and registers them. It performs a reconciliation with the leader controller to kill any local containers if the unavailability was too long and controller has already spun up new alternatives.
+Executor service keeps track of all containers it is supposed to run by running periodic reconciliation with the leader controller. Any mismatch gets handled:
+An application is defined using JSON. We use a sample configuration below to explain the options.
+{
+ "name": "TEST_APP", // (1)!
+ "version": "1", // (2)!
+ "type": "SERVICE", // (3)!
+ "executable": { //(4)!
+ "type": "DOCKER", // (5)!
+ "url": "ghcr.io/appform-io/perf-test-server-httplib",// (6)!
+ "dockerPullTimeout": "100 seconds"// (7)!
+ },
+ "resources": [//(20)!
+ {
+ "type": "CPU",
+ "count": 1//(21)!
+ },
+ {
+ "type": "MEMORY",
+ "sizeInMB": 128//(22)!
+ }
+ ],
+ "volumes": [//(12)!
+ {
+ "pathInContainer": "/data",//(13)!
+ "pathOnHost": "/mnt/datavol",//(14)!
+ "mode" : "READ_WRITE"//(15)!
+ }
+ ],
+ "configs" : [//(16)!
+ {
+ "type" : "INLINE",//(17)!
+ "localFilename": "/testfiles/drove.txt",//(18)!
+ "data" : "RHJvdmUgdGVzdA=="//(19)!
+ }
+ ],
+ "placementPolicy": {//(23)!
+ "type": "ANY"//(24)!
+ },
+ "exposedPorts": [//(8)!
+ {
+ "name": "main",//(9)!
+ "port": 8000,//(10)!
+ "type": "HTTP"//(11)!
+ }
+ ],
+ "healthcheck": {//(25)!
+ "mode": {//(26)!
+ "type": "HTTP", //(27)!
+ "protocol": "HTTP",//(28)!
+ "portName": "main",//(29)!
+ "path": "/",//(30)!
+ "verb": "GET",//(31)!
+ "successCodes": [//(32)!
+ 200
+ ],
+ "payload": "", //(33)!
+ "connectionTimeout": "1 second" //(34)!
+ },
+ "timeout": "1 second",//(35)!
+ "interval": "5 seconds",//(36)!
+ "attempts": 3,//(37)!
+ "initialDelay": "0 seconds"//(38)!
+ },
+ "readiness": {//(39)!
+ "mode": {
+ "type": "HTTP",
+ "protocol": "HTTP",
+ "portName": "main",
+ "path": "/",
+ "verb": "GET",
+ "successCodes": [
+ 200
+ ],
+ "payload": "",
+ "connectionTimeout": "1 second"
+ },
+ "timeout": "1 second",
+ "interval": "3 seconds",
+ "attempts": 3,
+ "initialDelay": "0 seconds"
+ },
+ "exposureSpec": {//(42)!
+ "vhost": "testapp.local", //(43)!
+ "portName": "main", //(44)!
+ "mode": "ALL"//(45)!
+ },
+ "env": {//(41)!
+ "CORES": "8"
+ },
+ "args" : [//(54)!
+ "./entrypoint.sh",
+ "arg1",
+ "arg2"
+ ],
+ "tags": { //(40)!
+ "superSpecialApp": "yes_i_am",
+ "say_my_name": "heisenberg"
+ },
+ "preShutdown": {//(46)!
+ "hooks": [ //(47)!
+ {
+ "type": "HTTP",
+ "protocol": "HTTP",
+ "portName": "main",
+ "path": "/",
+ "verb": "GET",
+ "successCodes": [
+ 200
+ ],
+ "payload": "",
+ "connectionTimeout": "1 second"
+ }
+ ],
+ "waitBeforeKill": "3 seconds"//(48)!
+ },
+ "logging": {//(49)!
+ "type": "LOCAL",//(50)!
+ "maxSize": "100m",//(51)!
+ "maxFiles": 3,//(52)!
+ "compress": true//(53)!
+ }
+}
+SERVICE for an application/service.DOCKER.HTTP, HTTPS, TCP, UDP.READ_WRITE and READ_ONLYINLINE, EXECUTOR_LOCAL_FILE, CONTROLLER_HTTP_FETCH and EXECUTOR_HTTP_FETCH. Specifies how drove will get the contents to be injected..type specified above.ANY, ONE_PER_HOST, MATCH_TAG, NO_TAG, RULE_BASED, ANY and COMPOSITE. Rest of the parameters in this section will depend on the type.HTTP or CMD. Rest of the options in this example are HTTP specific.HTTP/HTTPSexposedPorts section.GET,PUT or POST.POST and PUT calls.exposedPorts section.ALL for now.UNREADY state during this time and hence won't have api calls routed to it via Drove Gateway.Right now Drove supports only docker containers. However as engines, both docker and podman are supported. Drove executors will fetch the executable directly from the registry based on the configuration provided.
+| Name | +Option | +Description | +
|---|---|---|
| Type | +type |
+Set type to DOCKER. |
+
| URL | +url |
+Docker container URL`. | +
| Timeout | +dockerPullTimeout |
+Timeout for docker image pull. | +
Note
+Drove supports docker registry authentication. This can be configured in the executor configuration file.
+This section specifies the hardware resources required to run the container. Right now only CPU and MEMORY are supported as resource types that can be reserved for a container.
+Specifies number of cores to be assigned to the container.
+| Name | +Option | +Description | +
|---|---|---|
| Type | +type |
+Set type to CPU for this. |
+
| Count | +count |
+Number of cores to be assigned. | +
Specifies amount of memory to be allocated to a container.
+| Name | +Option | +Description | +
|---|---|---|
| Type | +type |
+Set type to MEMORY for this. |
+
| Count | +sizeInMB |
+Amount of memory (in Mega Bytes) to be allocated. | +
Sample +
[
+ {
+ "type": "CPU",
+ "count": 1
+ },
+ {
+ "type": "MEMORY",
+ "sizeInMB": 128
+ }
+]
+Note
+Both CPU and MEMORY configurations are mandatory.
Files and directories can be mounted from the executor host into the container. The volumes section contains a list of volumes that need to be mounted.
| Name | +Option | +Description | +
|---|---|---|
| Path In Container | +pathInContainer |
+Path that will be visible inside the container for this mount. | +
| Path On Host | +pathOnHost |
+Actual path on the host machine for the mount. | +
| Mount Mode | +mode |
+Mount mode can be READ_WRITE and READ_ONLY to allow the containerized process to write or read to the volume. |
+
Info
+We do not support mounting remote volumes as of now.
+Drove supports injection of configuration files into containers. The specifications for the same are discussed below.
+Inline configuration can be added in the Application Specification itself. This will manifest as a file inside the container.
+The following details are needed for this:
+| Name | +Option | +Description | +
|---|---|---|
| Type | +type |
+Set the value to INLINE |
+
| Local Filename | +localFilename |
+File name for the config inside the container. | +
| Data | +data |
+Base64 encoded string for the data. The value for this will be masked on UI. | +
Config file: +
port: 8080
+logLevel: DEBUG
+{
+ "type" : "INLINE",
+ "localFilename" : "/config/service.yml",
+ "data" : "cG9ydDogODA4MApsb2dMZXZlbDogREVCVUcK"
+}
+Warning
+The full base 64 encoded config data will get stored in Drove ZK and will be pushed to executors inline. It is not recommended to stream large config files to containers using this method. This will probably need additional configuration on your ZK cluster.
+Config file from a path on the executor directly. Such files can be distributed to the executor host using existing configuration management systems such as OpenTofu, Salt etc.
+The following details are needed for this:
+| Name | +Option | +Description | +
|---|---|---|
| Type | +type |
+Set the value to EXECUTOR_LOCAL_FILE |
+
| Local Filename | +localFilename |
+File name for the config inside the container. | +
| File path | +filePathOnHost |
+Path to the config file on executor host. | +
Sample config specification: +
{
+ "type" : "EXECUTOR_LOCAL_FILE",
+ "localFilename" : "/config/service.yml",
+ "data" : "/mnt/configs/myservice/config.yml"
+}
+Config file can be fetched from a remote server by the controller. Once fetched, these will be streamed to the executor as part of the instance specification for starting a container.
+The following details are needed for this:
+| Name | +Option | +Description | +
|---|---|---|
| Type | +type |
+Set the value to CONTROLLER_HTTP_FETCH |
+
| Local Filename | +localFilename |
+File name for the config inside the container. | +
| HTTP Call Details | +http |
+HTTP Call related details. Please refer to HTTP Call Specification for details. | +
Sample config specification: +
{
+ "type" : "CONTROLLER_HTTP_FETCH",
+ "localFilename" : "/config/service.yml",
+ "http" : {
+ "protocol" : "HTTP",
+ "hostname" : "configserver.internal.yourdomain.net",
+ "port" : 8080,
+ "path" : "/configs/myapp",
+ "username" : "appuser",
+ "password" : "secretpassword"
+ }
+}
+Note
+The controller will make an API call for every single time it asks an executor to spin up a container. Please make sure to account for this in your configuration management system.
+Config file can be fetched from a remote server by the executor before spinning up a container. Once fetched, the payload will be injected as a config file into the container.
+The following details are needed for this:
+| Name | +Option | +Description | +
|---|---|---|
| Type | +type |
+Set the value to EXECUTOR_HTTP_FETCH |
+
| Local Filename | +localFilename |
+File name for the config inside the container. | +
| HTTP Call Details | +http |
+HTTP Call related details. Please refer to HTTP Call Specification for details. | +
Sample config specification: +
{
+ "type" : "EXECUTOR_HTTP_FETCH",
+ "localFilename" : "/config/service.yml",
+ "http" : {
+ "protocol" : "HTTP",
+ "hostname" : "configserver.internal.yourdomain.net",
+ "port" : 8080,
+ "path" : "/configs/myapp",
+ "username" : "appuser",
+ "password" : "secretpassword"
+ }
+}
+Note
+All executors will make an API call for every single time they spin up a container for this application. Please make sure to account for this in your configuration management system.
+This section details the options that can set when making http calls to a configuration management system from controllers or executors.
+The following options are available for HTTP call:
+| Name | +Option | +Description | +
|---|---|---|
| Protocol | +protocol |
+Protocol to use for upstream call. Can be HTTP or HTTPS. |
+
| Hostname | +hostname |
+Host to call. | +
| Port | +port |
+Provide custom port. Defaults to 80 for http and 443 for https. | +
| API Path | +path |
+Path component of the URL. Include query parameters here. Defaults to / |
+
| HTTP Method | +verb |
+Type of call, use GET, POST or PUT. Defaults to GET. |
+
| Success Code | +successCodes |
+List of HTTP status codes which is considered as success. Defaults to [200] |
+
| Payload | +payload |
+Data to be used for POST and PUT calls | +
| Connection Timeout | +connectionTimeout |
+Timeout for upstream connection. | +
| Operation timeout | +operationTimeout |
+Timeout for actual operation. | +
| Username | +username |
+Username to be used basic auth. This field is masked out on the UI. | +
| Password | +password |
+Password to be used for basic auth. This field is masked on the UI. | +
| Authorization Header | +authHeader |
+Data to be passed in HTTP Authorization header. This field is masked on the UI. |
+
| Additional Headers | +headers |
+Any other headers to be passed to the upstream in the HTTP calls. This is a map of | +
| Skip SSL Checks | +insecure |
+Skip hostname and certification checks during SSL handshake with the upstream. | +
Placement policy governs how Drove deploys containers on the cluster. The following sections discuss the different placement policies available and how they can be configured to achieve optimal placement of containers.
+Warning
+All policies will work only at a {appName, version} combination level. They will not ensure constraints at an appName level. This means that for somethinge like a one per node placement, for the same appName, multiple containers can run on the same host if multiple deployments with different versions are active in a cluster. Same applies for all policies like N per host and so on.
Important details about executor tagging
+TAG policy will consider them as valid tags. This can be used to place containers on specific hosts if needed.MATCH_TAG policyContainers for a {appName, version} combination can run on any un-tagged executor host.
| Name | +Option | +Description | +
|---|---|---|
| Policy Type | +type |
+Put ANY as policy. |
+
Sample: +
{
+ "type" : "ANY"
+}
+Tip
+For most use-cases this is the placement policy to use.
+Ensures that only one container for a particular {appName, version} combination is running on an executor host at a time.
| Name | +Option | +Description | +
|---|---|---|
| Policy Type | +type |
+Put ONE_PER_HOST as policy. |
+
Sample: +
{
+ "type" : "ONE_PER_HOST"
+}
+Ensures that at most N containers for a {appName, version} combination is running on an executor host at a time.
| Name | +Option | +Description | +
|---|---|---|
| Policy Type | +type |
+Put MAX_N_PER_HOST as policy. |
+
| Max count | +max |
+The maximum num of containers that can run on an executor. Range: 1-64 | +
Sample: +
{
+ "type" : "MAX_N_PER_HOST",
+ "max": 3
+}
+Ensures that containers for a {appName, version} combination are running on an executor host that has the tags as mentioned in the policy.
| Name | +Option | +Description | +
|---|---|---|
| Policy Type | +type |
+Put MATCH_TAG as policy. |
+
| Max count | +tag |
+The tag to match. | +
Sample: +
{
+ "type" : "MATCH_TAG",
+ "tag": "gpu_enabled"
+}
+Ensures that containers for a {appName, version} combination are running on an executor host that has no tags.
| Name | +Option | +Description | +
|---|---|---|
| Policy Type | +type |
+Put NO_TAG as policy. |
+
Sample: +
{
+ "type" : "NO_TAG"
+}
+Info
+The NO_TAG policy is mostly for internal use, and does not need to be specified when deploying containers that do not need any special placement logic.
+Composite policy can be used to combine policies together to create complicated placement requirements.
+| Name | +Option | +Description | +
|---|---|---|
| Policy Type | +type |
+Put COMPOSITE as policy. |
+
| Polices | +policies |
+List of policies to combine | +
| Combiner | +combiner |
+Can be AND and OR and signify all-match and any-match logic on the policies mentioned. |
+
Sample: +
{
+ "type" : "COMPOSITE",
+ "policies": [
+ {
+ "type": "ONE_PER_HOST"
+ },
+ {
+ "type": "MATH_TAG",
+ "tag": "gpu_enabled"
+ }
+ ],
+ "combiner" : "AND"
+}
+{appName,version} will run on GPU enabled machines.
+Tip
+It is easy to go into situations where no executors match complicated placement policies. Internally, we tend to keep things rather simple and use the ANY placement for most cases and maybe tags in a few places with over-provisioning or for hosts having special hardware 
This config can be used to inject custom environment variables to containers. The values are defined as part of deployment specification, are same across the cluster and immutable to modifications from inside the container (ie any overrides from inside the container will not be visible across the cluster).
+Sample: +
{
+ "MY_VARIABLE_1": "fizz",
+ "MY_VARIABLE_2": "buzz"
+}
+The following environment variables are injected by Drove to all containers:
+| Variable Name | +Value | +
|---|---|
| HOST | +Hostname where the container is running. This is for marathon compatibility. | +
PORT_PORT_NUMBER |
+A variable for every port specified in exposedPorts section. The value is the actual port on the host, the specified port is mapped to. For example if ports 8080 and 8081 are specified, two variables called PORT_8080 and PORT_8081 will be injected. |
+
| DROVE_EXECUTOR_HOST | +Hostname where container is running. | +
| DROVE_CONTAINER_ID | +Container that is deployed | +
| DROVE_APP_NAME | +App name as specified in the Application Specification | +
| DROVE_INSTANCE_ID | +Actual instance ID generated by Drove | +
| DROVE_APP_ID | +Application ID as generated by Drove | +
| DROVE_APP_INSTANCE_AUTH_TOKEN | +A JWT string generated by Drove that can be used by this container to call /apis/v1/internal/... apis. |
+
Warning
+Do not pass secrets using environment variables. These variables are all visible on the UI as is. Please use Configs to inject secrets files and so on.
+A list of command line arguments that are sent to the container engine to execute inside the container. This is provides ways for you to configure your container behaviour based off such arguments. Please refer to docker documentation for details.
+Danger
+This might have security implications from a system point of view. As such Drove provides administrators a way to disable passing arguments at the cluster level by setting disableCmdlArgs to true in the controller configuration.
One of the cornerstones of managing applications on the cluster is to ensure we keep track of instance health and manage their life cycle depending on their health state. We need to define how to monitor health for containers accordingly. The checks will be executed on Applications and a Check result is generated. The result consists of the following:
+| Name | +Option | +Description | +
|---|---|---|
| Mode | +mode |
+The definition of a HTTP call or a Command to be executed in the container. See following sections for details. | +
| Timeout | +timeout |
+Duration for which we wait before declaring a check as failed | +
| Interval | +interval |
+Interval at which check will be retried | +
| Attempts | +attempts |
+Number of times a check is retried before it is declared as a failure | +
| Initial Delay | +initialDelay |
+Delay before executing the check for the first time. | +
Note
+initialDelay is ignored when readiness checks and health checks are run in the recovery path as the container is already running at that point in time.
| Name | +Option | +Description | +
|---|---|---|
| Type | +type |
+Fixed to HTTP for HTTP checker | +
| Protocol | +protocol |
+HTTP or HTTPS call to be made | +
| Port Name | +portName |
+The name of the container port to make the http call on as specified in the Exposed Ports section in Application Spec | +
| Path | +path |
+The api path to call | +
| HTTP method | +verb |
+The HTTP Verb/Method to invoke. GET/PUT and POST are supported here | +
| Success Codes | +successCodes |
+A set of HTTP status codes that we should consider as a success from this API. | +
| Payload | +payload |
+A string payload that we can pass if the Verb is POST or PUT | +
| Connection Timeout | +connectionTimeout |
+Maximum time for which the checker will wait for the connection to be set up with the container. | +
| Insecure | +insecure |
+Skip hostname and certificate checks for HTTPS ports during checks. | +
| Field | +Option | +Description | +
|---|---|---|
| Type | +type |
+Fixed to CMD for command checker | +
| Command | +command |
+Command to execute in the container. (Equivalent to docker exec -it <container> command>) |
+
Exposure spec is used to specify the virtual host Drove Gateway exposes to outside world for communication with the containers.
+The following information needs to be specified:
+| Name | +Option | +Description | +
|---|---|---|
| Virtual Host | +vhost |
+The virtual host to be exposed on NGinx. This should be a fully qualified domain name. | +
| Port Name | +portName |
+The portname to be exposed on the vhost. Port names are defined in exposedPorts section. |
+
| Exposure Mode | +mode |
+Use ALL here for now. Signifies that all healthy instances of the app are exposed to traffic. |
+
Sample: +
{
+ "vhost": "teastapp.mydomain",
+ "port": "main",
+ "mode": "ALL"
+}
+Note
+Application instances in any state other than HEALTHY are not considered for exposure. Please check Application Instance State Machine for an understanding of states of instances.
Before a container is shut down, it is desirable to ensure things are spun down properly. This behaviour can be configured in the preShutdown section of the configuration.
| Name | +Option | +Description | +
|---|---|---|
| Hooks | +hooks |
+List of api calls and commands to be run on the container before it is killed. Each hook is either a HTTP Call Spec or Command Spec | +
| Wait Time | +waitBeforeKill |
+Time to wait before killing the container. | +
Sample +
{
+ "hooks": [
+ {
+ "type": "HTTP",
+ "protocol": "HTTP",
+ "portName": "main",
+ "path": "/",
+ "verb": "GET",
+ "successCodes": [
+ 200
+ ],
+ "payload": "",
+ "connectionTimeout": "1 second"
+ }
+ ],
+ "waitBeforeKill": "3 seconds"//(48)!
+}
+Note
+The waitBeforeKill timed wait kicks in after all the hooks have been executed.
Can be used to configure how container logs are managed on the system.
+Note
+This section affects the docker log driver. Drove will continue to stream logs to it's own logger which can be configured at executor level through the executor configuration file.
+This is used to configure the json-file log driver.
| Name | +Option | +Description | +
|---|---|---|
| Type | +type |
+Set the value to LOCAL |
+
| Max Size | +maxSize |
+Maximum file size. Anything bigger than this will lead to rotation. | +
| Max Files | +maxFiles |
+Maximum number of logs files to keep. Range: 1-100 | +
| Compress | +compress |
+Enable log file compression. | +
Tip
+If logging section is omitted, the following configuration is applied by default:
+- File size: 10m
+- Number of files: 3
+- Compression: on
In case suers want to stream logs to an rsyslog server, the logging configuration needs to be set to RSYSLOG mode.
+| Name | +Option | +Description | +
|---|---|---|
| Type | +type |
+Set the value to RSYSLOG |
+
| Server | +server |
+URL for the rsyslog server. | +
| Tag Prefix | +tagPrefix |
+Prefix to add at the start of a tag | +
| Tag Suffix | +tagSuffix |
+Suffix to add at the en of a tag. | +
Note
+The default tag is the DROVE_INSTANCE_ID. The tagPrefix and tagSuffix will to before and after this