human password login benchmark of zitadel v4

Benchmark results of v4 release of Zitadel.

This benchmark tests the flow of a user signing in to Zitadel using username and password which is a highly CPU intensive task for checking the password.

Performance test results

Metric	Value
Baseline	none
Purpose	Test current performance
Test start	06:43 UTC
Test duration	30min
Executed test	human_password_login
k6 version	v1.0.0-rc1
VUs	600
Client location	US1
ZITADEL location	US1
ZITADEL container specification	vCPU: 6 Memory: 6 Gi Container min scale: 2 Container max scale: 7
ZITADEL Version	v4.0.0-rc2
ZITADEL feature flags	webKey: true, improvedPerformance: ["IMPROVED_PERFORMANCE_ORG_BY_ID", "IMPROVED_PERFORMANCE_PROJECT", "IMPROVED_PERFORMANCE_USER_GRANT", "IMPROVED_PERFORMANCE_ORG_DOMAIN_VERIFIED", "IMPROVED_PERFORMANCE_PROJECT_GRANT"]
Database	type: psql version: v17.4
Database location	US1
Database specification	vCPU: 8 memory: 32Gib
ZITADEL metrics during test
Observed errors
Top 3 most expensive database queries	1: Locking projections for reduce 2: Query events by aggregate type and event type 3: Query events by instance id, aggregate types, event types, position
k6 Iterations per second	39
k6 output	output
flowchart outcome

Endpoint latencies

Until :45 were containers starting because it was a cold start test.

k6 output

  █ TOTAL RESULTS 

    checks_total.......................: 867495 464.541713/s
    checks_succeeded...................: 99.88% 866490 out of 867495
    checks_failed......................: 0.11%  1005 out of 867495

    ✓ user defined
    ✗ authorize status ok
      ↳  99% — ✓ 72470 / ✗ 435
    ✗ login name status ok
      ↳  99% — ✓ 72331 / ✗ 139
    ✓ login shows password page
    ✗ password status ok
      ↳  99% — ✓ 72032 / ✗ 155
    ✗ password callback
      ↳  99% — ✓ 72000 / ✗ 32
    ✓ code set
    ✗ token status ok
      ↳  99% — ✓ 71866 / ✗ 134
    ✓ access token created
    ✓ id token created
    ✓ info created
    ✓ org created
    ✓ create user is status ok
    ✗ openid configuration
      ↳  98% — ✓ 600 / ✗ 11
    ✗ userinfo status ok
      ↳  99% — ✓ 71755 / ✗ 99
    ✓ org removed

    CUSTOM
    human_password_login_duration...........................................: min=984ms    avg=14.73s   max=58.88s  p(50)=12.41s   p(95)=36.8s    p(99)=40.55s  
    login_ui_enter_login_name_duration......................................: min=98.28ms  avg=10.57s   max=38.72s  p(50)=5.99s    p(95)=32.4s    p(99)=36.16s  
    login_ui_enter_password_duration........................................: min=1.12ms   avg=119.12ms max=1m0s    p(50)=2.15ms   p(95)=3.96ms   p(99)=3.8s    
    login_ui_init_login_duration............................................: min=41.73ms  avg=600.61ms max=22.51s  p(50)=292.69ms p(95)=1.84s    p(99)=2.82s   
    login_ui_token_duration.................................................: min=53.03ms  avg=712.39ms max=21.76s  p(50)=339.01ms p(95)=2.11s    p(99)=3.32s   
    oidc_user_info_duration.................................................: min=14.54ms  avg=187.13ms max=21.51s  p(50)=74.47ms  p(95)=398.78ms p(99)=619.27ms
    org_create_org_duration.................................................: min=50.09ms  avg=50.09ms  max=50.09ms p(50)=50.09ms  p(95)=50.09ms  p(99)=50.09ms 
    user_create_human_duration..............................................: min=425.26ms avg=11.83s   max=16.89s  p(50)=13.67s   p(95)=16.07s   p(99)=16.78s  

    HTTP
    http_req_duration.......................................................: min=1.12ms   avg=1.87s    max=1m0s    p(50)=257.41ms p(95)=12.28s   p(99)=31.18s  
      { expected_response:true }............................................: min=1.12ms   avg=1.84s    max=38.72s  p(50)=256.14ms p(95)=11.83s   p(99)=31.19s  
    http_req_failed.........................................................: 0.19%  1117 out of 581385
    http_reqs...............................................................: 581385 311.330421/s

    EXECUTION
    iteration_duration......................................................: min=983.33ms avg=14.88s   max=1m11s   p(50)=12.64s   p(95)=36.85s   p(99)=40.62s  
    iterations..............................................................: 72904  39.039936/s
    vus.....................................................................: 18     min=0              max=600
    vus_max.................................................................: 600    min=600            max=600

    NETWORK
    data_received...........................................................: 1.2 GB 658 kB/s
    data_sent...............................................................: 235 MB 126 kB/s

running (31m07.4s), 000/600 VUs, 72904 complete and 0 interrupted iterations