Skip to content

Latest commit

 

History

History
1098 lines (686 loc) · 71.1 KB

performance-test-report.md

File metadata and controls

1098 lines (686 loc) · 71.1 KB

Performance Test Report

Introduction

This document provides details on the performance measurement approach of prioritized scenarios of MOSIP modules and provides detailed reports on the results.

Scope

APIs in all modules are prioritized based on the high load expectation and usage frequency by other modules.

There are 3 types of performance scenarios covered in this report

  • Prioritized single API performance (For most modules)
  • End to end UI API sequence performance (For pre-registration module)
  • End to end message delivery performance (For web-sub module)

All individual test scenarios from the modules are listed below.

Module Name API Scenario API Endpoint
Kernel Add Audits v1/auditmanager/audits
Kernel Client Id - Secret Key v1/authmanager/authenticate/clientidsecretkey
Kernel User Id Pwd v1/authmanager/authenticate/internal/useridPwd
Kernel Validate Token v1/authmanager/authorize/admin/validateToken
Kernel Generate OTP v1/otpmanager/otp/generate
Kernel Validate OTP v1/otpmanager/otp/validate?key={keyValidate}&otp={otpValidate}
Kernel Send SMS v1/notifier/sms/send
Kernel Send EMAIL v1/notifier/email/send
Kernel Generate PRID v1/pridgenerator/prid
Kernel Generate UIN v1/idgenerator/uin
Kernel Generate VID v1/idgenerator/vid
Kernel Generate RID v1/ridgenerator/generate/rid/{centerID}/{machineID}
IDA Auth with OTP idauthentication/v1/auth/{mispLicenseKey}/{rpPartnerId}/{rpApiKey}
IDA KYC with OTP idauthentication/v1/kyc/{mispLicenseKey}/{ekycPartnerId}/{ekycApiKey}
IDA Auth with bio idauthentication/v1/auth/{mispLicenseKey}/{rpPartnerId}/{rpApiKey}
IDA KYC with bio idauthentication/v1/kyc/{mispLicenseKey}/{ekycPartnerId}/{ekycApiKey}
IDA Send OTP idauthentication/v1/otp/{mispLicenseKey}/{rpPartnerId}/{rpApiKey}
Pre-Registration Prereg UI end to end flow Prereg UI end to end flow
Syncdata Public Key Verify v1/syncdata/tpm/publickey/verify
Syncdata Get Certificate v1/syncdata/getCertificate?applicationId={appID}&referenceId={refID}
Syncdata Get User Details v1/syncdata/userdetails?keyindex={keyIndex}
Syncdata Get Client Settings v1/syncdata/v2/clientsettings?keyindex={keyIndex}
Syncdata Get Configs v1/syncdata/configs/{name}
Syncdata Get LatestID Schema v1/syncdata/latestidschema?schemaVersion={schemaVersion}
Syncdata Get CAcertificates v1/syncdata/getcacertificates
Regproc Sync Registration Packet Details registrationprocessor/v1/registrationstatus/sync
Regproc Get Packet status registrationprocessor/v1/registrationstatus/search
Regproc Upload Registration Packet registrationprocessor/v1/packetreceiver/registrationpackets
Resident Request/Send OTP resident/v1/req/otp
Resident RID Check Status resident/v1/rid/check-status
Resident Auth Lock resident/v1/req/auth-lock
Resident Auth Unlock resident/v1/req/auth-unlock
Resident Credential Request resident/v1/req/credential
Resident Auth History resident/v1/req/auth-history
Resident Generate VID resident/v1/vid
Datashare Create Data Share URL v1/datashare/create/{policyId}/{subscriberId}
Datashare Get Data Share File v1/datashare/get/{policyId}/{subscriberId}/{randomShareKey}
IDRepo Retrieve Identity using UIN idrepository/v1/identity/idvid/{uin}
IDRepo Retrieve Identity using VID idrepository/v1/identity/idvid/{vid}
IDRepo Add Identity idrepository/v1/identity/
IDRepo Update Identity idrepository/v1/identity/
IDRepo Auth Type Status idrepository/v1/identity/authtypes/status
IDRepo Create VID idrepository/v1/vid
IDRepo Update VID idrepository/v1/vid/{vidGenerated}
Masterdata Get Registration Centers v1/masterdata/registrationcenters
Masterdata Get Templates v1/masterdata/templates
Masterdata Get Templates By Lang & Temp Code v1/masterdata/templates/{lang&TempTypeCode}
Masterdata Get Latest ID Schema v1/masterdata/idschema/latest?schemaVersion={schemaVersion}
Masterdata Get Dynamic Fields v1/masterdata/dynamicfields?pageNumber=0&pageSize=10&sortBy=name&orderBy=desc
Masterdata Get Users v1/masterdata/users/{userID}/{time(yyyy-MM-dd'T'HH:mm:ss.sss'Z',)}
Keymanager Encrypt Data v1/keymanager/encrypt
Keymanager Encrypt Data With Pin v1/keymanager/encryptWithPin
Keymanager Generate JWT Signature v1/keymanager/jwtSign
Keymanager Decrypt Data v1/keymanager/decrypt
Keymanager Decrypt Data With Pin API v1/keymanager/decryptWithPin
Keymanager Verify JWT Signature v1/keymanager/jwtVerify

Testing Tools & Techniques

Tools Purpose
Jmeter To verify the Load and Performance of all the applications.

Approach

The approach to performance testing differs based on the testing scenarios and are detailed as per the categorization as below.

Prioritized Single API Performance

  • Run a single pod of the application hosting the required API
  • Run four pods each of the dependent applications to cater the load implied from the test application
  • Simulate varied load profilers like 10, 30, 50 etc.. concurrent users from Jmeter for that single API for a period of one hour
  • Record the average and 90th percentile response time metrics specific to each load profiles
  • Verify that JVMs show stable memory and CPU usage over the increasing load profiles.

End to End UI API Sequence Performance

  • Run a single pod for each application hosting the required APIs called directly from Jmeter
  • Run four pods each of the dependent applications to cater the load implied from the test application
  • Simulate varied load profilers like 50, 80, 100 etc.. concurrent users from Jmeter for end to end sequence of APIs called from UI for a period of one hour
  • Record the average and 90th percentile response time metrics for entire end to end API sequence for each load profiles
  • Verify that JVMs show stable memory and CPU usage over the increasing load profiles.

End to End Message Delivery Performance

  • Run a single pod for each application hosting the required APIs called directly from Jmeter
  • Run four pods each of the dependent applications to cater the load implied from the test application
  • Simulate varied load profilers like 10, 30, 50 etc.. concurrent users to publish continues messages to the test application for a period of one hour
  • Record the average and 90th percentile response time metrics for publish API for each load profiles
  • Record the average and 90th percentile delivery time metrics for entire end to end message delivery for each load profiles

Build Details

Title Description
Build Tag Version 1.2.0-rc2
Performance script source location https://github.com/mosip/mosip-performance-tests-mt/tree/1.2.0

Environment Setup

Hardware / VM details

CPU (cores) Memory (GB) HDD (GB) OS Count Purpose
8 30 768 Cent OS 1 Kubernetes console machine
8 Kubernetes node machines
4 16 500 Ubuntu 1 Database machine
4 16 835 Windows 3 Jmeter test machines

Pod Resource allocation

Pod/Application Names Resource Allocation (Limits)
Memory (M) CPU (m) Heap (M)
kernel-auditmanager-service 2500 1000 1750
kernel-auth-service 2250 500 1500
kernel-otpmanager-service 1750 500 1000
kernel-notification-service 1500 500 1000
kernel-pridgenerator-service 1750 300 1000
kernel-idgenerator-service 3500 500 2000
kernel-ridgenerator-service 1750 500 1000
ida-auth-service 4000 4500 2000
ida-otp-service 3000 3000 2000
prereg-application-service 2500 1000 2000
kernel-syncdata-service 5000 500 4000
regproc-registration-status-service 4000 500 2000
regproc-stage-group-1 5000 1000 4000
resident-service 3000 500 2000
datashare-service 3000 500 2000
idrepo-identity-service 4000 1000 2000
idrepo-vid-service 3000 500 1000
kernel-masterdata-service 2250 500 1500
kernel-keymanager-service 5000 1000 4000

Test Results

Prioritized single API performance

90th Percentile Response Times

Detailed response time details

Detailed Metrics for all 14 modules is available below:

Audit Manager

Run 1: 50 concurrency

Jmeter Aggregate Report

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Add Audit API Execution 231236 777 395 2206 2700 3402 3 8420 0.00% 64.22209 38.88 186.99

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

Dependent Application Response Time Graph from Kibana

Dependency app graph for auth manager service was empty so not attached.

Run 2: 30 concurrency

Jmeter Aggregate Report

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Add Audit API Execution 266879 403 105 1199 1893 2409 2 3870 0.00% 74.12657 44.88 215.82

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

Dependent Application Response Time Graph from Kibana

Application dependency graph was empty so not attached.

Auth Manager

Run 1: 30 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Client Id - Secret Key API Execution 241902 449 202 483 597 3352 9 60051 0.16% 66.1073 290.08 28.49
User Id Pwd API Execution 220878 488 488 680 711 872 70 6616 0.00% 61.34721 215.9 30
Validate Token API Execution 451378 239 201 400 496 680 6 3506 0.00% 125.3778 582.61 235.51

Response Time Percentile Graph using Jmeter

We only have the graph for Validate token as the previous ones were cleared

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

Dependency app graph was empty so so not attached.

Notes:

Keycloak was having issue by end of client id secret key API run, so the keycloak was restarted before the start of user id password API

Run 2: 50 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Client Id - Secret Key API Execution 243069 740 499 1694 2100 2833 9 7511 0.00% 67.51412 296.36 29.09
User Id Pwd API Execution 27121 5519 3290 11676 22351 43894 40 60047 0.39% 9.05834 31.82 4.42
Validate Token API Execution 181953 989 810 2107 2496 3206 6 8111 0.00% 50.53694 234.74 94.93

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

Dependent Application Response Time Graph from Kibana

Dependency app graph was empty so so not attached.

Notes:

Keycloak was having issue by end of user id password API run, so the keycloak was restarted before the start of validate token API

Kernel services

Run 1: 30 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Generate OTP API Execution 87777 1229 1104 2160 2496 3207 5 7418 0.00% 24.37841 15.57 48.04
Validate OTP API Execution 150000 1275 1128 2192 2404 3196 0 6268 0.02% 23.4176 14.89 42.63
Notification Manager SMS API Request 433004 246 99 692 808 1119 2 2625 0.00% 120.2695 80.92 259.67
Notification Manager EMAIL API Request 69727 1550 1396 2992 3602 5101 4 13607 0.04% 14.87585 9.48 41.37
PRID Generator API Execution 102606 1052 1022 1893 2073 2432 5 5104 0.00% 28.49246 11.74 51
Generate UIN API Execution 39971 2698 2672 2813 2833 3536 127 8776 0.00% 11.09746 26.27 23.71
Generate VID API Execution 205377 154 139 266 321 423 5 12380 6.39% 193.1971 458.89 412.75
RID Generator API Execution 368802 291 193 743 930 1324 5 5761 0.00% 102.4346 63.42 221.07

Note: idgenerator- generate vid api threw error as “VID not available for allocation error”, so it ran only for around 20 mins rest all other api’s ran for the complete 1 hour run.

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

Note: idgenerator & pridgenerator services are vertx apps so there are no kibana graphs for the same

Dependent Application Response Time Graph from Kibana

Dependency app graph was empty so above are the graphs for the application services from kibana

Notes: Some of the kernel service api’s failed with oomkilled & authentication failed issues, so restarted the same and then continued the load test from the next set of api’s that’s why we have multiple graphs from jmeter & grafana.

Run 2: 10 & 50 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Generate OTP API Execution 768397 45 9 92 95 100 5 299 0.00% 213.4406 135.9 420.63
Generate OTP API Request 90000 42 8 90 91 94 4 184 0.00% 225.8061 143.77 445
Validate OTP API Execution 90000 48 14 92 95 100 6 1209 0.00% 201.6066 127.78 366.99
Notification Manager EMAIL API Request 135462 264 195 606 893 1198 3 6299 0.00% 37.62565 23.92 104.68
Notification Manager SMS API Request 4085605 23 1 2 2 8 1 60063 99.89% 1134.873 414.17 2450.28
PRID Generator API Execution 77782 462 459 792 853 949 8 2303 0.00% 21.60354 8.86 38.67
RID Generator API Execution 1881518 86 16 80 97 136 1 60184 99.91% 516.4455 1062.7 1114.59
Generate UIN API Execution 41290 870 875 896 905 924 87 1785 0.00% 11.46714 27.12 24.5
Generate VID API Execution 1132665 157 114 330 355 396 4 3560 87.76% 314.6231 763.26 672.26

Note: idgenerator- generate vid api threw error as “VID not available for allocation error”, so it ran only for around 10 mins & also notification manager sms api & rid generator api failed completely with 502 bad gateway errors for 50 concurrency.

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Note: The grafana graphs for the other services like otpmanager, ridgenerator, pridgenerator & id generator services are lost as they were old data

Application Response Time Graph from Kibana

Note: idgenerator & pridgenerator services are vertx apps so there are no kibana graphs for the same

Dependent Application Response Time from Kibana

Dependency app graph was empty so above are the graphs for the application services from kibana

Notes: Some of the kernel service api’s failed with oomkilled & authentication failed issues, so restarted the same and then continued the load test from the next set of api’s that’s why we have multiple graphs from jmeter & grafana.

IDA

Run 1: 10 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Send OTP Execution 15743 2286 2396 2804 2933 3192 25 3855 0.01% 4.371 12.22 19.2
ID Auth Request With OTP 23060 1560 1547 2058 2192 2424 175 3185 0.02% 6.40415 17.91 33.45
EKYC Auth Request With OTP 22252 1616 1614 2129 2265 2500 294 3286 0.02% 6.17927 794.05 32.46
ID Auth Request With Biometrics 21804 1649 1639 2200 2343 2611 252 3794 0.02% 6.05471 16.93 220.35
EKYC Auth Request With Biometrics 21042 1709 1691 2285 2434 2728 210 3712 0.02% 5.84423 750.99 212.86

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Grafana graphs were erased for older data

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

Run 2: 30 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Send OTP Execution 13576 7958 7995 8996 9308 10123 4293 12046 0.03% 3.76589 10.54 16.55
ID Auth Request With OTP 20202 5348 5353 6020 6261 6805 2174 8238 0.02% 5.60468 15.68 29.28
EKYC Auth Request With OTP 19889 5430 5445 6142 6402 6949 1027 8252 0.02% 5.51885 709.21 28.99
ID Auth Request With Biometrics 19403 5568 5581 6357 6639 7252 670 9007 0.02% 5.38179 15.06 195.86
EKYC Auth Request With Biometrics 18949 5702 5726 6538 6853 7509 353 8850 0.02% 5.25693 675.52 191.47

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Grafana graphs were erased for older data.

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

Registration Processor

Run 1: 30 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Sync Registration Packet API Execution 53535 2017 1878 3029 3519 4997 199 14424 0.00% 14.86683 15.96 51.03
Get Packet status API Execution 87599 1232 1233 1552 1630 1776 243 60013 0.00% 24.32718 181.49 63.17
Upload Registration Packet API Execution 3510 30911 30991 33291 34363 38198 478 53099 0.00% 0.96944 0.86 1948.02

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

No data was captured for registration processor registration status service graph in Kibana for the entire test duration.

Note: Registration processor stage group 1 service is a vertx app so there is no Kibana graph for this.

Dependent Application Response Time from Kibana

Run 2: 10 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Sync Registration Packet API Execution 64649 556 550 676 708 909 207 2110 0.00% 17.95645 19.27 61.64
Get Packet status API Execution 49530 726 703 978 1036 1133 169 60012 0.00% 13.7564 145.97 35.72
Upload Registration Packet API Execution 6690 5386 1260 11404 12258 14306 123 18888 0.00% 1.85473 1.65 3727.21

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

No data was captured for registration processor registration status service graph in Kibana for the entire test duration.

Note: registration processor stage group 1 service is a vertx app, so there is no kibana graph for this.

Dependent Application Response Time from Kibana

Syncdata

Run 1: 30 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Public Key Verify API Execution 435249 247 209 458 574 866 21 5613 0.00% 120.8957 171.66 338.84
Get Certificate API Execution 353275 305 269 546 654 891 20 2089 0.00% 98.12497 276.26 181.01
Get User Details API Execution 11736 9213 9166 9695 10276 14573 730 22247 0.00% 3.25181 11.79 6.16
Get Client Settings API Execution 48931 2207 1999 3703 4397 5803 196 10803 0.00% 13.58629 2368.66 25.82
Get Configs API Execution 20427 5290 5193 5911 6286 7203 1106 10596 0.00% 5.66731 262.18 10.24
Get LatestID Schema API Execution 356781 302 218 651 836 1276 20 3355 0.00% 99.10027 1510.41 179.43
Get CAcertificates API Execution 238505 452 369 878 1101 1648 28 4755 0.00% 66.24493 7158.97 119.1

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

Run 2: 10 & 50 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Public Key Verify API Execution 390125 460 355 947 1206 1815 21 6124 0.00% 108.3583 153.65 303.7
Get Certificate API Execution 300097 599 500 1172 1429 2005 20 5102 0.00% 83.34947 234.5 153.76
Get User Details API Execution 11773 3058 3054 3205 3261 3534 833 5279 0.00% 3.2678 11.84 6.19
Get Client Settings API Execution 52127 690 692 902 1001 1199 79 1900 0.00% 14.478 2524.09 27.51
Get Configs API Execution 18717 1923 1806 2206 2386 3310 1006 4100 0.00% 5.19704 240.42 9.39
Get LatestID Schema API Execution 298200 603 416 1369 1780 2732 21 7979 0.00% 82.82468 1262.19 149.96
Get CA certificates API Execution 220333 816 599 1747 2248 3424 28 8297 0.00% 61.19382 6613.03 110.02

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

Resident

Run 1: 30 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Request OTP 5277 20527 13618 60012 60013 60021 1325 60079 11.29% 1.45586 1.33 2.92
RID Check Status 73109 1477 1013 3220 4260 6770 105 21521 0.00% 20.29957 12.91 40.1
Auth Lock API 16981 6364 5670 10923 12893 17213 567 31744 0.00% 4.71056 3.19 9.53
Auth Unlock API 16223 6662 5966 11496 13537 17909 375 33340 0.00% 4.49985 3.06 9.26
Request Credential API 16253 6652 5994 12227 14217 19044 273 38494 0.00% 4.50174 2.95 9.83
Auth History API 5100 21221 18707 37493 44213 59170 1479 60023 0.82% 1.41057 0.99 2.89
Generate VID API Execution 14410 7504 6832 12998 15335 20769 600 37105 0.00% 3.99359 2.67 8.06

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

CPU container usage grafana graphs are not available for resident service container

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

Run 2: 10 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Request OTP 4810 7493 7549 8397 8635 9405 1463 13833 0.00% 1.33296 0.93 2.67
RID Check Status 56043 641 562 1101 1379 1959 103 3881 0.00% 15.56573 9.91 30.75
Auth Lock API 10000 2411 2010 4085 4872 6406 258 9587 0.02% 4.12136 2.79 8.34
Auth Unlock API 10000 2438 2060 4165 4900 6349 401 9044 0.02% 4.08215 2.77 8.4
Request Credential API 10000 2440 1557 5310 6421 9416 215 13339 0.02% 4.0734 2.67 8.9
Auth History API 4490 8025 7987 9591 10157 11535 2059 14581 0.00% 1.24482 0.85 2.55
Generate VID API Execution 10000 2720 2090 5090 6213 8163 385 11400 0.02% 3.6507 2.45 7.37

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

CPU container usage grafana graphs are not available for resident service container

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

Datashare

Run 1: 30 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Create Data Share URL Execution 13434 8044 7901 11306 12400 14202 1421 18805 0.00% 3.72685 3.44 3845.7
Get Data Share File Execution 185315 582 302 804 2775 4705 8 12710 0.00% 51.47058 4263.17 115.79

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

Run 2: 10 & 50 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Create Data Share URL Execution 13253 2716 2528 4035 4532 5426 249 12256 0.00% 3.6798 3.4 3797.15
Get Data Share File Execution 179001 1004 602 2498 2899 3520 10 7101 0.00% 49.71529 33.6 111.84

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

ID Repository

Run 1: 30 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Add Identity (bio) Execution 9000 7407 7366 9210 9878 11094 110 15147 27.29% 4.02176 2.88 4397.92
Retrieve Identity using UIN 505883 213 198 295 358 697 22 1304 0.22% 140.5188 243.91 303.41
Retrieve Identity using VID 130678 825 506 1701 1913 2452 13 3792 0.46% 36.29708 62.91 78.59
Create VID Execution 105575 1022 798 1992 2296 2501 99 3618 0.48% 29.32075 20.84 69.64
Update Identity Execution 15000 3167 3085 4707 5207 6402 121 9115 2.49% 9.3834 6.7 34.79
Update VID Execution 15000 1455 1403 2388 2496 2699 5 3516 0.67% 20.48912 15.76 49.04

Note: Add Identity api threw a lot of duplicate key constraint errors ~27.29 %

\norg.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "uk_uin_reg_id"

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

No data was captured for the Kibana graphs of id repository identity & VID services

Dependent Application Response Time from Kibana

Run 2: 10 & 50 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Retrieve Identity using UIN 389893 461 313 994 1209 1598 28 2317 0.22% 108.2907 187.86 233.82
Retrieve Identity using VID 110457 325 191 800 914 1194 12 1983 0.46% 30.67593 53.14 66.42
Add Identity (bio) Execution 15000 2868 2901 3402 3750 4196 140 4896 26.35% 3.46978 2.48 3794.32
Update Identity Execution 20000 1269 1328 1973 2276 2900 14 4586 3.86% 7.84082 5.57 29.07
Auth Type Status Execution 306901 586 418 1198 1496 1892 33 2702 0.00% 85.2427 53.28 226.51
Create VID Execution 77102 466 302 1002 1181 1513 45 2271 0.48% 21.41616 15.2 50.86
Update VID Execution 13950 576 423 1138 1282 1612 0 2377 0.51% 17.23997 13.23 41.26

Note: Add Identity API threw a lot of duplicate key constraint errors ~27.29 %

\norg.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "uk_uin_reg_id"

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

No data was captured for the Kibana graphs of ID repository identity & VID services

Dependent Application Response Time from Kibana

Masterdata

Run 1: 30 & 50 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Get Registration Centres API Execution 21480 5030 4901 7806 9098 10898 253 17595 0.00% 5.96056 3609.27 10.74
Get Templates API Execution 170957 1051 802 2505 2992 4294 4 10242 0.00% 47.46712 4540.57 85.06
Get Templates By Lang & Temp Code API Execution 47241 2285 2002 3885 4555 5526 4 9282 0.00% 13.11962 16.77 23.87
Get Latest ID Schema API Execution 69179 2601 2501 4388 5200 6573 5 10700 0.00% 19.20952 279.19 34.84
Get Dynamic Fields API Execution 55536 1944 1884 2893 3098 3416 14 5094 0.00% 15.42405 81.76 28.45
Get Users API Execution 44399 2432 2295 4196 4597 5785 6 9189 0.00% 12.3283 9.31 22.42

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

No data was captured for the Grafana - CPU & memory container usage graphs

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

No data was captured for dependency apps in kibana

Run 2: 10 & 30 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Get Registration Centres API Execution 2281 15799 15772 18399 19296 21013 1904 23509 0.00% 0.63243 382.95 1.14
Get Templates API Execution 93851 1149 1198 2190 2533 2997 4 5791 0.00% 26.0657 2493.37 46.71
Get Templates By Lang & Temp Code API Execution 46088 780 699 1300 1403 1592 4 2905 0.00% 12.79908 16.36 23.29
Get Latest ID Schema API Execution 68743 1570 1399 2694 2865 3896 5 5604 0.00% 19.09314 277.5 34.62
Get Dynamic Fields API Execution 58867 611 489 1113 1200 1301 11 1531 0.00% 16.34786 86.66 30.16
Get Users API Execution 43642 824 901 1320 1475 1595 6 2602 0.00% 12.12249 9.15 22.04

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

No data was captured in Grafana for CPU & memory container usage graphs

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

No data was captured for dependency apps in Kibana.

Key Manager

Run 1: 30 & 50 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Encrypt Data API Execution 1053790 170 189 287 297 386 6 1604 0.00% 292.7122 317.26 753.15
Encrypt Data With Pin API Execution 10443 10346 10406 13892 14705 16399 1589 19495 0.00% 2.89775 2.2 6.92
Generate JWT Signature API Execution 558466 321 310 437 485 798 14 1579 0.00% 155.117 417.01 392.79
Decrypt Data API Execution 516634 348 335 475 528 858 14 1442 0.00% 143.4985 93.17 428.67
Decrypt Data With Pin API Execution 10349 10442 10515 14208 15288 17010 1704 21202 0.00% 2.87069 1.86 7.13
Verify JWT Signature API Execution 1582654 113 90 209 305 628 3 1404 0.00% 439.616 297.94 1995.87

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

No data was captured in Grafana for CPU & memory container usage graphs.

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

No data was captured for dependency apps in Kibana.

Run 2: 10 & 80 concurrency

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Encrypt Data API Execution 927329 310 292 583 707 930 6 1901 0.00% 257.552 279.65 662.68
Encrypt Data With Pin API Execution 10361 3475 3401 4305 4694 5296 711 6699 0.00% 2.87609 2.19 6.87
Generate JWT Signature API Execution 521059 552 531 669 895 1146 17 2150 0.00% 144.718 388.77 366.46
Decrypt Data API Execution 488850 588 549 827 1051 1366 15 4124 0.00% 135.7712 88.16 405.59
Decrypt Data With Pin API Execution 10089 3568 3500 4402 4699 5299 1109 6497 0.00% 2.80138 1.82 6.96
Verify JWT Signature API Execution 1214824 236 95 602 808 1334 3 2998 0.00% 337.423 229.34 1531.91

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana

No data was captured for dependency apps in kibana.

End to End UI API Sequence Performance Metrics

Pre-registration

Run 1: 50 concurrency

90th Percentile response time ( End to end UI flow) - 18.9 sec

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
PreReg Launch 2481 14 6 29 57 151 4 612 0.00% 0.68942 2.5 0.34
PreReg Send OTP 2481 5400 3910 12575 16399 24523 34 45492 0.00% 0.68841 0.48 0.29
PreReg Validate OTP 2480 3369 2189 8182 11242 19216 49 34813 0.00% 0.68807 2.15 1.16
PreReg Enter Demographic Details And Click Continue 2473 17520 15587 30322 34981 45715 1175 72550 0.00% 0.68645 23.3 6.15
PreReg Upload Documents 2463 7707 6095 15331 18711 28729 392 41042 0.00% 0.68532 3.17 371.05
PreReg Click Continue After Uploading Documents 2455 14692 13030 26844 31673 40660 243 56219 0.00% 0.68426 7.93 3.53
PreReg Preview Document Details And Continue 2446 14298 12314 25695 30528 38385 1667 54085 0.00% 0.68189 8.23 2.21
PreReg Select Center And Continue 2438 3410 2213 8151 10711 17372 65 32880 0.00% 0.68227 41.99 1.32
PreReg Select Slot And Continue 2438 6837 5402 13687 17525 25248 201 38337 5.41% 0.68167 4.19 2.97
PreReg Logout 2431 17 11 45 66 84 7 442 0.00% 0.68089 2.45 0.87
TOTAL 24586 7328 4598 18901 24527 35440 4 72550 0.54% 6.81687 95.81 388.5

Note: Error percentage for pre-registration is due to the unavailability of slots for 53 unique registration centres.

Slots were generated for 947 unique reg centres and for the rest 53 centres slots were not created so this needs to be debugged further.

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Self-response time graphs from Kibana

Dependent Application Response Time from Kibana

Notes: Prereg is connected to the internal database because when we tried connecting it to the external database we were getting failures while booking appointments manually.

Run 2: 80 concurrency

90th Percentile response time ( End to end UI flow) - 30.9 sec

Aggregate Report from Jmeter

Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
PreReg Launch 2389 66 9 183 412 707 4 1084 0.00% 0.66437 2.4 0.33
PreReg Send OTP 2389 8777 6358 19932 25454 34345 31 60013 0.08% 0.66207 0.46 0.28
PreReg Validate OTP 2385 5905 4286 12937 16567 28790 60 46664 0.00% 0.661 2.3 1.12
PreReg Enter Demographic Details And Click Continue 2381 29540 26824 50104 56729 70781 3687 99983 0.00% 0.66017 22.78 5.89
PreReg Upload Documents 2358 13605 11462 26031 32216 46567 752 62849 0.04% 0.6566 3.04 355.5
PreReg Click Continue After Uploading Documents 2350 24769 21697 42821 53048 76185 764 118903 0.04% 0.65785 7.62 3.39
PreReg Preview Document Details And Continue 2336 22505 19701 39685 47397 67283 1633 96086 0.13% 0.65444 7.89 2.12
PreReg Select Center And Continue 2319 5791 3802 13131 18470 29293 63 60365 0.04% 0.65156 39.94 1.26
PreReg Select Slot And Continue 2316 11313 9361 21756 27996 39136 303 61613 5.70% 0.65044 3.99 2.83
PreReg Logout 2309 139 25 499 648 986 8 1387 0.00% 0.65443 2.36 0.84
TOTAL 23532 12254 7990 30921 39493 57968 4 118903 0.60% 6.51774 91.88 371.52

Note:

  • Error percentage for pre-registration is due to the unavailability of slots for 53 unique reg centres. Slots were generated for 947 unique reg centres and for the rest 53 centres slots were not created so this needs to be debugged further.
  • Got some 504 gateway timeout errors for a few of the transactions before booking an appointment.

Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana

Self-response time graphs from Kibana

Dependent Application Response Time from Kibana

Notes: Pre-registration is connected to the internal database because when we tried connecting it to an external database we were getting failures while booking appointments manually.

End to End Message Delivery Performance Metrics

30 concurrency

  • 90th Percentile response time
    • Publish (10 subscribers, 8 topics) - 0.221 sec
    • End to end message delivery (10 subscribers, 8 topics) -
      • finalAvgNinetiethPercentile=5777.9 sec finalAvgTurnAroundTime=3327.8 sec
      • Aggregate Report from Jmeter
  • Aggregate Report from Jmeter
Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Publish Messages 605668 177 173 221 239 295 5 1759 0.00% 168.2339 52.74 333.1
  • Response Time Percentile Graph using Jmeter

50 concurrency

  • 90th Percentile response time
    • Publish (10 subscribers, 8 topics) - 0.25 sec
    • End to end message delivery (10 subscribers, 8 topics)
      • finalAvgNinetiethPercentile = 9810 sec
      • finalAvgTurnAroundTime = 5719.9 sec
  • Aggregate Report from Jmeter
Label # Samples Average Median 90% Line 95% Line 99% Line Min Max Error % Throughput Received KB/sec Sent KB/sec
Publish Messages 861179 208 202 252 270 316 4 2083 0.00% 239.2059 74.99 473.62
  • Response Time Percentile Graph using Jmeter