Skip to content

hical: pin HICAL_REF to 2a45016 (latest main) + trigger re-benchmark - #1057

Open
Hical61 wants to merge 12 commits into
MDA2AV:mainfrom
Hical61:feat/hical-update-commit
Open

hical: pin HICAL_REF to 2a45016 (latest main) + trigger re-benchmark#1057
Hical61 wants to merge 12 commits into
MDA2AV:mainfrom
Hical61:feat/hical-update-commit

Conversation

@Hical61

@Hical61 Hical61 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Description

Pin HICAL_REF from main to commit 2a45016, ensuring reproducible builds.

Since the initial PR (#864, Jun 13), Hical's main branch has accumulated 41 commits, including performance-relevant changes:

  • 6062f54 perf: optimistic sync write + early readBuf release (throughput improvement under high concurrency)
  • 633d181 perf: speculative read on stack buffer, eliminating ~20k unnecessary epoll_ctl calls
  • 0894b2b perf: idempotent preparePayload + WebSocket frame buffer reuse (P99 latency improvement)
  • 7416731 perf: ChunkedBody/SSE/Gzip compression optimizations
  • 06a4768 feat: PerfectHashRouter — compile-time perfect hash for static route lookup
  • 071ce4c feat: GzipCompression middleware (v2.6.6, enables json-comp)

Changes

  • frameworks/hical/Dockerfile: Switch from git clone --branch main to git init + fetch + checkout FETCH_HEAD, pinning HICAL_REF to 2a45016bbdcbf3dd93838beaf058e109f3f5dbfd

Trigger re-benchmark to reflect the latest code on the leaderboard.


/benchmark -f hical --save

@Hical61 Hical61 changed the title hical: pin HICAL_REF to latest main (2a45016) + trigger re-benchmark hical: pin HICAL_REF to 2a45016 (latest main) + trigger re-benchmark Jul 27, 2026
@MDA2AV

MDA2AV commented Jul 27, 2026

Copy link
Copy Markdown
Owner

/benchmark -f hical --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: hical | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 2,914,656 6591.9% 47MiB +1.3% -6.0%
baseline 4096 3,169,820 6634.5% 114MiB +4.1% -19.1%
pipelined 512 4,494,550 6550.9% 42MiB +8.5% +5.0%
pipelined 4096 4,510,973 6618.6% 146MiB +11.1% -2.0%
limited-conn 512 1,992,732 6271.8% 66MiB -0.7% -27.5%
limited-conn 4096 2,280,937 6474.9% 110MiB -0.1% -31.2%
json 4096 666,298 6655.0% 211MiB -5.7% -6.6%
json-comp 512 263,577 6356.7% 114MiB +0.1% -5.0%
json-comp 4096 262,443 6540.4% 234MiB -0.3% +1.7%
json-comp 16384 257,400 6317.1% 591MiB -0.8% +1.5%
upload 32 1,834 2301.1% 5.0GiB +2.9% +11.1%
upload 256 1,716 6302.6% 9.2GiB +0.5% +1.1%
static 1024 22,692 5833.4% 180MiB -81.6% +10.4%
static 4096 22,476 5851.4% 195MiB -81.3% -63.8%
static 6800 22,442 5740.5% 186MiB -81.0% -78.2%
echo-ws 512 3,376,333 6617.8% 49MiB +3.2% +8.9%
echo-ws 4096 3,738,094 6577.0% 158MiB +2.2% -4.2%
echo-ws 16384 3,368,423 6527.9% 510MiB +3.9% +0.6%
Full log
    Latency    151us     85us    177us   2.34ms   5.09ms

  16881667 frames sent in 5.00s, 16881667 frames received
  Throughput: 3.38M req/s
  Bandwidth:  22.53MB/s
  WS upgrades: 512
  WS frames:   16881667
  Latency samples: 16881609 / 16881667 responses (100.0%)
[info] CPU 6617.8% | Mem 49MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    151us     90us    190us   1.76ms   5.07ms

  16845581 frames sent in 5.00s, 16845581 frames received
  Throughput: 3.37M req/s
  Bandwidth:  22.49MB/s
  WS upgrades: 512
  WS frames:   16845581
  Latency samples: 16845535 / 16845581 responses (100.0%)
[info] CPU 6570.4% | Mem 49MiB

=== Best: 3376333 req/s (CPU: 6617.8%, Mem: 49MiB) ===
[info] saved results/echo-ws/512/hical.json
httparena-bench-hical
httparena-bench-hical

==============================================
=== hical / echo-ws / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.09ms    549us   3.52ms   6.71ms   9.74ms

  18599434 frames sent in 5.00s, 18595532 frames received
  Throughput: 3.72M req/s
  Bandwidth:  24.92MB/s
  WS upgrades: 4094
  WS frames:   18595532
  Latency samples: 18595532 / 18595532 responses (100.0%)
[info] CPU 6282.3% | Mem 147MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.09ms    559us   3.53ms   6.72ms   9.77ms

  18690802 frames sent in 5.00s, 18690470 frames received
  Throughput: 3.74M req/s
  Bandwidth:  24.96MB/s
  WS upgrades: 4096
  WS frames:   18690470
  Latency samples: 18691056 / 18690470 responses (100.0%)
[info] CPU 6577.0% | Mem 158MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.10ms    558us   3.54ms   6.66ms   9.61ms

  18653253 frames sent in 5.00s, 18653238 frames received
  Throughput: 3.73M req/s
  Bandwidth:  24.90MB/s
  WS upgrades: 4096
  WS frames:   18654617
  Latency samples: 18653154 / 18653238 responses (100.0%)

  WARNING: 18446744073709550237/18653238 responses (98892986159880.4%) had unexpected status (expected 2xx)
[info] CPU 6430.9% | Mem 162MiB

=== Best: 3738094 req/s (CPU: 6577.0%, Mem: 158MiB) ===
[info] saved results/echo-ws/4096/hical.json
httparena-bench-hical
httparena-bench-hical

==============================================
=== hical / echo-ws / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.78ms   5.19ms   8.51ms   11.80ms   56.10ms

  16464564 frames sent in 5.00s, 16448325 frames received
  Throughput: 3.29M req/s
  Bandwidth:  22.35MB/s
  WS upgrades: 16381
  WS frames:   16448325
  Latency samples: 16448325 / 16448325 responses (100.0%)
[info] CPU 5986.5% | Mem 491MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.83ms   5.28ms   8.49ms   11.40ms   15.10ms

  16858502 frames sent in 5.00s, 16842118 frames received
  Throughput: 3.37M req/s
  Bandwidth:  22.88MB/s
  WS upgrades: 16384
  WS frames:   16842118
  Latency samples: 16842118 / 16842118 responses (100.0%)
[info] CPU 6527.9% | Mem 510MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.83ms   5.27ms   8.54ms   11.40ms   14.60ms

  16788037 frames sent in 5.00s, 16771654 frames received
  Throughput: 3.35M req/s
  Bandwidth:  22.79MB/s
  WS upgrades: 16383
  WS frames:   16771654
  Latency samples: 16771654 / 16771654 responses (100.0%)
[info] CPU 6252.3% | Mem 516MiB

=== Best: 3368423 req/s (CPU: 6527.9%, Mem: 510MiB) ===
[info] saved results/echo-ws/16384/hical.json
httparena-bench-hical
httparena-bench-hical
[info] skip: hical does not subscribe to echo-ws-pipeline
[info] skip: hical does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/hical.json - 18 new, 18 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536

@Hical61

Hical61 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

CI benchmark results look uneven — the static file test shows a major regression (RPS dropped from ~120K to ~22K, -81%), likely caused by one of the 41 commits between the pinned refs. We'll investigate the root cause, fix it, and trigger another benchmark run after that.

@Hical61

Hical61 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Fixed the static file regression. Pinned HICAL_REF to be8bf5d — ready for another run.

/benchmark -f hical --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: hical | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 2,860,187 6679.2% 48MiB -0.6% -4.0%
baseline 4096 3,047,947 6648.0% 116MiB ~0% -17.7%
pipelined 512 4,343,257 6539.0% 42MiB +4.8% +5.0%
pipelined 4096 4,459,287 6371.3% 138MiB +9.8% -7.4%
limited-conn 512 1,977,959 6250.4% 64MiB -1.4% -29.7%
limited-conn 4096 2,222,053 6468.9% 112MiB -2.7% -30.0%
json 4096 666,206 6652.1% 209MiB -5.7% -7.5%
json-comp 512 260,427 6407.5% 129MiB -1.1% +7.5%
json-comp 4096 259,799 6556.2% 228MiB -1.3% -0.9%
json-comp 16384 257,000 6317.2% 595MiB -0.9% +2.2%
upload 32 1,821 2395.7% 4.9GiB +2.1% +8.9%
upload 256 1,710 6310.1% 8.7GiB +0.2% -4.4%
static 1024 24,343 5786.4% 187MiB -80.2% +14.7%
static 4096 23,801 5703.2% 203MiB -80.2% -62.3%
static 6800 23,968 6103.0% 205MiB -79.7% -76.0%
echo-ws 512 3,344,497 6626.1% 49MiB +2.2% +8.9%
echo-ws 4096 3,671,610 6577.8% 156MiB +0.4% -5.5%
echo-ws 16384 3,302,536 6535.4% 513MiB +1.9% +1.2%
Full log
    Latency    152us     86us    173us   2.30ms   5.12ms

  16722485 frames sent in 5.00s, 16722485 frames received
  Throughput: 3.34M req/s
  Bandwidth:  22.32MB/s
  WS upgrades: 512
  WS frames:   16722485
  Latency samples: 16722410 / 16722485 responses (100.0%)
[info] CPU 6626.1% | Mem 49MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    155us     89us    184us   2.45ms   4.30ms

  16443859 frames sent in 5.00s, 16443857 frames received
  Throughput: 3.29M req/s
  Bandwidth:  21.95MB/s
  WS upgrades: 512
  WS frames:   16443858
  Latency samples: 16443792 / 16443857 responses (100.0%)

  WARNING: 18446744073709551615/16443857 responses (112180153802782.1%) had unexpected status (expected 2xx)
[info] CPU 6583.3% | Mem 49MiB

=== Best: 3344497 req/s (CPU: 6626.1%, Mem: 49MiB) ===
[info] saved results/echo-ws/512/hical.json
httparena-bench-hical
httparena-bench-hical

==============================================
=== hical / echo-ws / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.11ms    567us   3.55ms   6.69ms   9.76ms

  18269866 frames sent in 5.00s, 18266707 frames received
  Throughput: 3.65M req/s
  Bandwidth:  24.46MB/s
  WS upgrades: 4095
  WS frames:   18266707
  Latency samples: 18266702 / 18266707 responses (100.0%)
[info] CPU 6450.6% | Mem 146MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.11ms    562us   3.55ms   6.70ms   9.62ms

  18358294 frames sent in 5.00s, 18358052 frames received
  Throughput: 3.67M req/s
  Bandwidth:  24.51MB/s
  WS upgrades: 4096
  WS frames:   18358052
  Latency samples: 18357945 / 18358052 responses (100.0%)
[info] CPU 6577.8% | Mem 156MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.12ms    569us   3.55ms   6.72ms   9.70ms

  18332194 frames sent in 5.00s, 18332139 frames received
  Throughput: 3.67M req/s
  Bandwidth:  24.47MB/s
  WS upgrades: 4096
  WS frames:   18332139
  Latency samples: 18332076 / 18332139 responses (100.0%)
[info] CPU 6405.0% | Mem 163MiB

=== Best: 3671610 req/s (CPU: 6577.8%, Mem: 156MiB) ===
[info] saved results/echo-ws/4096/hical.json
httparena-bench-hical
httparena-bench-hical

==============================================
=== hical / echo-ws / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.90ms   5.29ms   8.55ms   11.80ms   63.50ms

  15975084 frames sent in 5.00s, 15958844 frames received
  Throughput: 3.19M req/s
  Bandwidth:  21.70MB/s
  WS upgrades: 16383
  WS frames:   15958844
  Latency samples: 15958844 / 15958844 responses (100.0%)
[info] CPU 5916.0% | Mem 491MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.89ms   5.11ms   8.80ms   17.70ms   23.50ms

  16529065 frames sent in 5.00s, 16512681 frames received
  Throughput: 3.30M req/s
  Bandwidth:  22.44MB/s
  WS upgrades: 16384
  WS frames:   16512681
  Latency samples: 16512681 / 16512681 responses (100.0%)
[info] CPU 6535.4% | Mem 513MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.98ms   5.36ms   8.54ms   11.70ms   15.20ms

  16356081 frames sent in 5.00s, 16339697 frames received
  Throughput: 3.27M req/s
  Bandwidth:  22.21MB/s
  WS upgrades: 16384
  WS frames:   16339697
  Latency samples: 16339697 / 16339697 responses (100.0%)
[info] CPU 6267.4% | Mem 520MiB

=== Best: 3302536 req/s (CPU: 6535.4%, Mem: 513MiB) ===
[info] saved results/echo-ws/16384/hical.json
httparena-bench-hical
httparena-bench-hical
[info] skip: hical does not subscribe to echo-ws-pipeline
[info] skip: hical does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/hical.json - 18 new, 18 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536

@Hical61

Hical61 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Scoped Gzip middleware to /json route group only — static file serving no longer goes through deflate. Ready for another run.

/benchmark -f hical --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@Hical61

Hical61 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Fixed missing — only has a forward declaration, the full definition is needed for . Ready for another run.

/benchmark -f hical --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: hical | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 3,085,111 6656.1% 47MiB +7.3% -6.0%
baseline 4096 3,309,843 6615.0% 115MiB +8.7% -18.4%
pipelined 512 4,742,356 6580.6% 41MiB +14.5% +2.5%
pipelined 4096 4,802,315 6601.2% 146MiB +18.3% -2.0%
limited-conn 512 2,098,165 6277.7% 66MiB +4.6% -27.5%
limited-conn 4096 2,412,451 6201.1% 90MiB +5.6% -43.8%
json 4096 660,912 6452.2% 214MiB -6.5% -5.3%
json-comp 512 259,278 6324.2% 114MiB -1.5% -5.0%
json-comp 4096 257,973 6555.7% 232MiB -2.0% +0.9%
json-comp 16384 252,791 6335.6% 526MiB -2.5% -9.6%
upload 32 1,801 2345.5% 4.8GiB +1.0% +6.7%
upload 256 1,726 6482.1% 7.7GiB +1.1% -15.4%
static 1024 118,262 1680.8% 180MiB -4.0% +10.4%
static 4096 74,457 1000.0% 572MiB -38.0% +6.3%
static 6800 60,444 812.8% 871MiB -48.8% +2.0%
echo-ws 512 3,314,100 6566.3% 48MiB +1.3% +6.7%
echo-ws 4096 3,671,290 6581.6% 157MiB +0.4% -4.8%
echo-ws 16384 3,287,818 6530.5% 511MiB +1.5% +0.8%
Full log

  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    155us     90us    189us   2.25ms   5.10ms

  16416239 frames sent in 5.00s, 16416239 frames received
  Throughput: 3.28M req/s
  Bandwidth:  21.91MB/s
  WS upgrades: 512
  WS frames:   16416239
  Latency samples: 16416178 / 16416239 responses (100.0%)
[info] CPU 6628.9% | Mem 49MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    153us     88us    167us   2.58ms   5.11ms

  16570502 frames sent in 5.00s, 16570502 frames received
  Throughput: 3.31M req/s
  Bandwidth:  22.12MB/s
  WS upgrades: 512
  WS frames:   16570502
  Latency samples: 16570446 / 16570502 responses (100.0%)
[info] CPU 6566.3% | Mem 48MiB

=== Best: 3314100 req/s (CPU: 6566.3%, Mem: 48MiB) ===
[info] saved results/echo-ws/512/hical.json
httparena-bench-hical
httparena-bench-hical

==============================================
=== hical / echo-ws / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.11ms    563us   3.55ms   6.72ms   9.76ms

  18240354 frames sent in 5.00s, 18237100 frames received
  Throughput: 3.65M req/s
  Bandwidth:  24.42MB/s
  WS upgrades: 4094
  WS frames:   18237100
  Latency samples: 18237096 / 18237100 responses (100.0%)
[info] CPU 6305.1% | Mem 146MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.11ms    576us   3.55ms   6.73ms   9.65ms

  18356501 frames sent in 5.00s, 18356451 frames received
  Throughput: 3.67M req/s
  Bandwidth:  24.51MB/s
  WS upgrades: 4096
  WS frames:   18356451
  Latency samples: 18356388 / 18356451 responses (100.0%)
[info] CPU 6581.6% | Mem 157MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.12ms    563us   3.56ms   6.68ms   9.62ms

  18327840 frames sent in 5.00s, 18326176 frames received
  Throughput: 3.66M req/s
  Bandwidth:  24.50MB/s
  WS upgrades: 4096
  WS frames:   18326176
  Latency samples: 18326156 / 18326176 responses (100.0%)
[info] CPU 6389.3% | Mem 162MiB

=== Best: 3671290 req/s (CPU: 6581.6%, Mem: 157MiB) ===
[info] saved results/echo-ws/4096/hical.json
httparena-bench-hical
httparena-bench-hical

==============================================
=== hical / echo-ws / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.89ms   5.26ms   8.51ms   11.70ms   63.10ms

  14662849 frames sent in 5.00s, 14646467 frames received
  Throughput: 2.93M req/s
  Bandwidth:  19.95MB/s
  WS upgrades: 16382
  WS frames:   14646467
  Latency samples: 14646467 / 14646467 responses (100.0%)
[info] CPU 5989.1% | Mem 491MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.96ms   5.35ms   8.57ms   11.40ms   15.30ms

  16455219 frames sent in 5.00s, 16439091 frames received
  Throughput: 3.29M req/s
  Bandwidth:  22.34MB/s
  WS upgrades: 16384
  WS frames:   16439091
  Latency samples: 16439091 / 16439091 responses (100.0%)
[info] CPU 6530.5% | Mem 511MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.96ms   5.34ms   8.59ms   11.90ms   15.30ms

  16397125 frames sent in 5.00s, 16380997 frames received
  Throughput: 3.28M req/s
  Bandwidth:  22.26MB/s
  WS upgrades: 16384
  WS frames:   16380997
  Latency samples: 16380997 / 16380997 responses (100.0%)
[info] CPU 6275.9% | Mem 517MiB

=== Best: 3287818 req/s (CPU: 6530.5%, Mem: 511MiB) ===
[info] saved results/echo-ws/16384/hical.json
httparena-bench-hical
httparena-bench-hical
[info] skip: hical does not subscribe to echo-ws-pipeline
[info] skip: hical does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/hical.json - 18 new, 18 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536

@Hical61

Hical61 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Gzip middleware is now scoped to the /json route group instead of being global — static file serving is back to normal, and baseline/pipelined/limited-conn also benefited since they no longer pass through the gzip middleware chain. json-comp saw a minor dip from the extra coroutine frame in route-group middleware, within noise. Ready to merge if this looks good.

Thanks for the patience running the benchmarks and helping debug the regression — much appreciated!

@Kaliumhexacyanoferrat

Kaliumhexacyanoferrat commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the patience running the benchmarks and helping debug the regression — much appreciated!

That is what we want to provide, no four-week-cycles as we had with TechEmpower - so don't hesitate to request benchmarks.

static file serving is back to normal

Short clarifying question, the benchmark still shows -50% for static/6800. Are you sure we should merge this?

@Hical61

Hical61 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Two more fixes landed:

  1. PathCache LRU promotion now uses sampling (1/256 hits) instead of grabbing unique_lock on every cache hit — eliminates the lock contention that caused -38%/-48% regression at 4096/6800 connections.
  2. Removed isTextMime branching — all files now go through setFileBody streaming send (64KB chunks, zero heap alloc). The text-file setBody path was left over from when gzip was global.

Ready for another run.

/benchmark -f hical --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: hical | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 3,148,986 6551.2% 48MiB +9.5% -4.0%
baseline 4096 3,298,252 6614.9% 113MiB +8.3% -19.9%
pipelined 512 4,821,856 6554.1% 40MiB +16.4% ~0%
pipelined 4096 4,805,495 6314.1% 136MiB +18.3% -8.7%
limited-conn 512 2,103,380 6241.2% 66MiB +4.8% -27.5%
limited-conn 4096 2,401,076 6235.3% 124MiB +5.1% -22.5%
json 4096 653,927 6648.3% 213MiB -7.5% -5.8%
json-comp 512 260,979 6349.9% 114MiB -0.9% -5.0%
json-comp 4096 258,969 6559.5% 229MiB -1.6% -0.4%
json-comp 16384 255,578 6186.4% 244MiB -1.5% -58.1%
upload 32 1,800 2337.9% 4.9GiB +1.0% +8.9%
upload 256 1,715 6511.6% 7.7GiB +0.5% -15.4%
static 1024 49,203 609.8% 122MiB -60.0% -25.2%
static 4096 17,729 271.5% 301MiB -85.2% -44.1%
static 6800 13,352 276.1% 433MiB -88.7% -49.3%
echo-ws 512 3,319,805 6559.5% 49MiB +1.5% +8.9%
echo-ws 4096 3,676,008 6574.0% 156MiB +0.5% -5.5%
echo-ws 16384 3,284,381 6286.6% 516MiB +1.3% +1.8%
Full log

  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    153us     90us    181us   2.28ms   4.34ms

  16587778 frames sent in 5.00s, 16587777 frames received
  Throughput: 3.32M req/s
  Bandwidth:  22.14MB/s
  WS upgrades: 512
  WS frames:   16587777
  Latency samples: 16587710 / 16587777 responses (100.0%)
[info] CPU 6616.0% | Mem 49MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    155us     93us    200us   1.94ms   4.43ms

  16448365 frames sent in 5.00s, 16448365 frames received
  Throughput: 3.29M req/s
  Bandwidth:  21.96MB/s
  WS upgrades: 512
  WS frames:   16448365
  Latency samples: 16448302 / 16448365 responses (100.0%)
[info] CPU 6564.5% | Mem 50MiB

=== Best: 3319805 req/s (CPU: 6559.5%, Mem: 49MiB) ===
[info] saved results/echo-ws/512/hical.json
httparena-bench-hical
httparena-bench-hical

==============================================
=== hical / echo-ws / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.11ms    564us   3.55ms   6.65ms   9.60ms

  18346155 frames sent in 5.00s, 18343334 frames received
  Throughput: 3.67M req/s
  Bandwidth:  24.55MB/s
  WS upgrades: 4096
  WS frames:   18343334
  Latency samples: 18343334 / 18343334 responses (100.0%)
[info] CPU 6344.5% | Mem 146MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.11ms    570us   3.56ms   6.66ms   9.63ms

  18380204 frames sent in 5.00s, 18380042 frames received
  Throughput: 3.67M req/s
  Bandwidth:  24.54MB/s
  WS upgrades: 4096
  WS frames:   18380042
  Latency samples: 18379972 / 18380042 responses (100.0%)
[info] CPU 6574.0% | Mem 156MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.11ms    567us   3.56ms   6.67ms   9.58ms

  18364561 frames sent in 5.00s, 18364368 frames received
  Throughput: 3.67M req/s
  Bandwidth:  24.52MB/s
  WS upgrades: 4096
  WS frames:   18364368
  Latency samples: 18364277 / 18364368 responses (100.0%)
[info] CPU 6390.6% | Mem 162MiB

=== Best: 3676008 req/s (CPU: 6574.0%, Mem: 156MiB) ===
[info] saved results/echo-ws/4096/hical.json
httparena-bench-hical
httparena-bench-hical

==============================================
=== hical / echo-ws / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.92ms   5.32ms   8.49ms   11.70ms   49.40ms

  15971120 frames sent in 5.00s, 15954879 frames received
  Throughput: 3.19M req/s
  Bandwidth:  21.70MB/s
  WS upgrades: 16381
  WS frames:   15954879
  Latency samples: 15954879 / 15954879 responses (100.0%)
[info] CPU 5979.7% | Mem 490MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.98ms   5.40ms   8.48ms   9.62ms   12.80ms

  16360852 frames sent in 5.00s, 16344468 frames received
  Throughput: 3.27M req/s
  Bandwidth:  22.22MB/s
  WS upgrades: 16384
  WS frames:   16344468
  Latency samples: 16344468 / 16344468 responses (100.0%)
[info] CPU 6537.8% | Mem 508MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.96ms   5.35ms   8.63ms   11.80ms   15.30ms

  16438291 frames sent in 5.00s, 16421907 frames received
  Throughput: 3.28M req/s
  Bandwidth:  22.32MB/s
  WS upgrades: 16384
  WS frames:   16421907
  Latency samples: 16421907 / 16421907 responses (100.0%)
[info] CPU 6286.6% | Mem 516MiB

=== Best: 3284381 req/s (CPU: 6286.6%, Mem: 516MiB) ===
[info] saved results/echo-ws/16384/hical.json
httparena-bench-hical
httparena-bench-hical
[info] skip: hical does not subscribe to echo-ws-pipeline
[info] skip: hical does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/hical.json - 18 new, 18 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536

@Hical61
Hical61 force-pushed the feat/hical-update-commit branch from e7a79a1 to 19af508 Compare July 28, 2026 13:04
@Hical61

Hical61 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

HICAL_REF pinned to bde1320 (restore isTextMime branching + lock-free thread_local PathCache).

Changes:

  1. Restore isTextMime() — text files go through setBody (2 coroutine suspensions, single scatter-gather write) instead of setFileBody (3 suspensions + sync open() blocking io_context)
  2. Lock-free thread_local PathCache — eliminates cross-core cache-line bouncing from the old global shared_mutex, expected to fix the -38%/-48% regression at 4096/6800 connections

Ready for another run.

/benchmark -f hical --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

1 similar comment
@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@Hical61

Hical61 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Fixed the commit hash typo in Dockerfile. Ready for another run.

/benchmark -f hical --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: hical | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 3,051,077 6565.2% 47MiB +6.1% -6.0%
baseline 4096 3,309,546 6609.2% 114MiB +8.7% -19.1%
pipelined 512 4,758,745 6548.1% 41MiB +14.9% +2.5%
pipelined 4096 4,824,190 6604.9% 145MiB +18.8% -2.7%
limited-conn 512 2,098,775 6339.0% 68MiB +4.6% -25.3%
limited-conn 4096 2,408,502 6431.4% 113MiB +5.4% -29.4%
json 4096 658,564 6485.6% 217MiB -6.8% -4.0%
json-comp 512 260,379 6360.9% 114MiB -1.1% -5.0%
json-comp 4096 262,094 6554.8% 230MiB -0.5% ~0%
json-comp 16384 257,019 6305.2% 611MiB -0.9% +5.0%
upload 32 1,814 2246.4% 5.3GiB +1.7% +17.8%
upload 256 1,727 6507.8% 8.2GiB +1.2% -9.9%
static 1024 127,768 1879.7% 195MiB +3.7% +19.6%
static 4096 73,505 938.3% 566MiB -38.8% +5.2%
static 6800 60,823 813.6% 891MiB -48.5% +4.3%
echo-ws 512 3,348,750 6571.4% 49MiB +2.4% +8.9%
echo-ws 4096 3,666,374 6570.5% 156MiB +0.2% -5.5%
echo-ws 16384 3,272,475 6529.5% 512MiB +1.0% +1.0%
Full log
    Latency    153us     87us    181us   2.29ms   6.05ms

  16650227 frames sent in 5.00s, 16650226 frames received
  Throughput: 3.33M req/s
  Bandwidth:  22.23MB/s
  WS upgrades: 512
  WS frames:   16650226
  Latency samples: 16656520 / 16650226 responses (100.0%)
[info] CPU 6635.0% | Mem 49MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    154us     88us    179us   2.48ms   5.13ms

  16493542 frames sent in 5.00s, 16493541 frames received
  Throughput: 3.30M req/s
  Bandwidth:  22.01MB/s
  WS upgrades: 512
  WS frames:   16493541
  Latency samples: 16493485 / 16493541 responses (100.0%)
[info] CPU 6550.6% | Mem 49MiB

=== Best: 3348750 req/s (CPU: 6571.4%, Mem: 49MiB) ===
[info] saved results/echo-ws/512/hical.json
httparena-bench-hical
httparena-bench-hical

==============================================
=== hical / echo-ws / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.11ms    565us   3.56ms   6.70ms   9.87ms

  18239412 frames sent in 5.00s, 18236210 frames received
  Throughput: 3.65M req/s
  Bandwidth:  24.42MB/s
  WS upgrades: 4096
  WS frames:   18236210
  Latency samples: 18236204 / 18236210 responses (100.0%)
[info] CPU 6315.1% | Mem 147MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.12ms    570us   3.55ms   6.72ms   9.79ms

  18330720 frames sent in 5.00s, 18331872 frames received
  Throughput: 3.66M req/s
  Bandwidth:  24.47MB/s
  WS upgrades: 4095
  WS frames:   18330541
  Latency samples: 18330485 / 18331872 responses (100.0%)

  WARNING: 1331/18331872 responses (0.0%) had unexpected status (expected 2xx)
[info] CPU 6570.5% | Mem 156MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.12ms    578us   3.56ms   6.65ms   9.62ms

  18326167 frames sent in 5.00s, 18325976 frames received
  Throughput: 3.66M req/s
  Bandwidth:  24.46MB/s
  WS upgrades: 4093
  WS frames:   18325976
  Latency samples: 18325899 / 18325976 responses (100.0%)
[info] CPU 6396.6% | Mem 160MiB

=== Best: 3666374 req/s (CPU: 6570.5%, Mem: 156MiB) ===
[info] saved results/echo-ws/4096/hical.json
httparena-bench-hical
httparena-bench-hical

==============================================
=== hical / echo-ws / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.93ms   5.34ms   8.50ms   11.60ms   46.30ms

  15878917 frames sent in 5.00s, 15862700 frames received
  Throughput: 3.17M req/s
  Bandwidth:  21.57MB/s
  WS upgrades: 16383
  WS frames:   15862700
  Latency samples: 15862700 / 15862700 responses (100.0%)
[info] CPU 5934.4% | Mem 491MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.96ms   5.36ms   8.63ms   11.60ms   15.20ms

  16378760 frames sent in 5.00s, 16362376 frames received
  Throughput: 3.27M req/s
  Bandwidth:  22.24MB/s
  WS upgrades: 16384
  WS frames:   16362376
  Latency samples: 16362376 / 16362376 responses (100.0%)
[info] CPU 6529.5% | Mem 512MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   4.98ms   5.38ms   8.55ms   11.60ms   15.10ms

  16332106 frames sent in 5.00s, 16315722 frames received
  Throughput: 3.26M req/s
  Bandwidth:  22.18MB/s
  WS upgrades: 16384
  WS frames:   16315722
  Latency samples: 16315722 / 16315722 responses (100.0%)
[info] CPU 6270.8% | Mem 516MiB

=== Best: 3272475 req/s (CPU: 6529.5%, Mem: 512MiB) ===
[info] saved results/echo-ws/16384/hical.json
httparena-bench-hical
httparena-bench-hical
[info] skip: hical does not subscribe to echo-ws-pipeline
[info] skip: hical does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/hical.json - 18 new, 18 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants