Skip to content

OSIS Initial Performance Analysis

Jump to bottom
Sai Tharun Thodupunuru edited this page Aug 30, 2021 · 31 revisions

Objective

Checking the performance behavior of the OSIS service on scality cloud. As OSIS is not configured through NGINX, only one node will be configured in vCloudDirector.

  • Tenant Flow: Query/Create/List Tenant

  • User Flow: Query/Get/Create/List User

  • Credential Flow: Query/Get/Create/List Credentials

Test Cases:

  • Base Case:

    • Simple flow → Query/Create/List Tenant, Query/Get/Create/List User, Query/Get/Create/List Credentials

  • Medium load:

    • 1 client (threads)

      • (loop count) 100 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 1000 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 10000 x Tenant Flow, User Flow, Credential Flow

    • 2 clients (threads)

      • (loop count) 100 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 1000 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 10000 x Tenant Flow, User Flow, Credential Flow

    • 4 clients (threads)

      • (loop count) 100 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 1000 x Tenant Flow, User Flow, Credential Flow

      • (loop count) 10000 x Tenant Flow, User Flow, Credential Flow

  • Ramp Up test - Stress failure test

    • Tenant Flow, User Flow, Credential Flow
Clients (threads) Loop Count(number of times to execute the test per thread) Rampup time: (Ramp-Up Period tells JMeter how long to delay before starting the next user) (in seconds)
128 10 300
256 10 300
512 10 300
128 10 600
256 10 600
512 10 600

Metrics to capture:

  • Elapsed time: Measures the elapsed time from just before sending the request to just after the last chunk of the response has been received,

  • Latency: Measures the latency from just before sending the request to just after the first chunk of the response has been received,

  • Median: Number which divides the samples into two equal halves,

  • 90% Line (90th Percentile): The elapsed time below which 90% of the samples fall

  • 95% Line (95th Percentile): The elapsed time below which 95% of the samples fall

  • Throughput: Calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. The formula is: Throughput = (number of requests) / (total time).

Storage Node metrics using RING grafana dashboards(http:///stats/):

  • Docker Image size

  • RAM/CPU/Disk usage

  • Report numbers to Product for tooling adjustments.

Test Results

1. Base Case:

Simple flow → Query/Create/List Tenant, Query/Get/Create/List User, Query/Get/Create/List Credentials

Service # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput
Create Tenant 1 1095 1095 1095 1095 1095 1095 1095 0.000% 0.91324
List Tenants 1 204 204 204 204 204 204 204 0.000% 4.90196
Query Tenants 1 205 205 205 205 205 205 205 0.000% 4.87805
Create User 1 1432 1432 1432 1432 1432 1432 1432 0.000% 0.69832
List Users 1 292 292 292 292 292 292 292 0.000% 3.42466
Query Users 1 266 266 266 266 266 266 266 0.000% 3.75940
Get User 1 236 236 236 236 236 236 236 0.000% 4.23729
Create Credential 1 281 281 281 281 281 281 281 0.000% 3.55872
List Credentials 1 319 319 319 319 319 319 319 0.000% 3.13480
Query Credentials 1 278 278 278 278 278 278 278 0.000% 3.59712
Get Credential 1 237 237 237 237 237 237 237 0.000% 4.21941
TOTAL 11 440 278 1095 1095 1432 204 1432 0.000% 2.23532

2. Medium load

i. 1 client (threads)

a.(loop count) 100 x Tenant Flow, User Flow, Credential Flow

Service # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput
Create Tenant 100 228 214 252 294 385 192 813 0.000% 0.38506
List Tenants 100 222 216 251 265 276 196 279 0.000% 0.38624
Query Tenants 100 203 198 221 228 262 185 296 0.000% 0.38622
Create User 100 386 374 464 494 564 292 565 0.000% 0.38603
List Users 100 210 206 239 244 281 185 307 0.000% 0.38630
Query Users 100 232 228 259 279 284 206 353 0.000% 0.38635
Get User 100 208 203 225 243 282 189 312 0.000% 0.38639
Create Credential 100 246 229 281 295 345 202 1253 0.000% 0.38644
List Credentials 100 227 223 256 269 287 200 294 0.000% 0.38643
Query Credentials 100 218 212 245 262 283 192 291 0.000% 0.38647
Get Credential 100 222 213 253 273 366 195 427 0.000% 0.38650
TOTAL 1100 237 217 304 373 478 185 1253 0.000% 4.19926

b.(loop count) 1000 x Tenant Flow, User Flow, Credential Flow

Service # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput
Service # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput
------------- ------------- ------------- ------------- ------------- ------------- ------------- ------------- ------------- ------------- -------------
Create Tenant 1000 200 195 218 232 266 178 706 0.000% 0.40013
List Tenants 1000 298 281 410 463 561 184 662 0.000% 0.40021
Query Tenants 1000 242 231 292 351 482 177 614 1.900% 0.40020
Create User 981 317 306 376 406 463 249 1190 0.000% 0.39257
List Users 981 191 187 206 218 246 176 298 0.000% 0.39273
Query Users 981 285 268 378 420 502 193 624 0.000% 0.39273
Get User 981 189 186 201 211 238 174 417 0.000% 0.39274
Create Credential 981 211 206 234 248 288 187 408 0.000% 0.39274
List Credentials 981 201 197 220 239 267 181 333 0.000% 0.39275
Query Credentials 981 197 193 214 226 256 179 320 0.000% 0.39276
Get Credential 981 196 192 213 223 248 178 313 0.000% 0.39277
TOTAL 10848 230 203 314 354 457 174 1190 0.175% 4.33657
Clone this wiki locally