Skip to content

Bring typescript benchmark client to parity with rust#4494

Merged
clockwork-labs-bot merged 1 commit intomasterfrom
noa/ts-benchmark-client
Mar 4, 2026
Merged

Bring typescript benchmark client to parity with rust#4494
clockwork-labs-bot merged 1 commit intomasterfrom
noa/ts-benchmark-client

Conversation

@coolreader18
Copy link
Collaborator

@coolreader18 coolreader18 commented Feb 27, 2026

Description of Changes

  • Enable pipelining by default
    • Set defaults for MAX_INFLIGHT_PER_WORKER for spacetime and convex
  • Add a warmup period
  • Reduce allocations in sdk (not that much of an effect but still improved things)

This gives improved parity with the rust client on my machine.

Expected complexity level and risk

1

Testing

  • Yup:
      RESULTS
    ══════════════════════════════════════════════════════════════════════
    
      spacetimedb    ████████████████████████████████████████     80,617 TPS
      convex         █░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░        235 TPS
    
      ╔════════════════════════════════════════════════════════════╗
      ║                                                            ║
      ║       🚀 spacetimedb is 343x FASTER than convex! 🚀        ║
      ║                                                            ║
      ╚════════════════════════════════════════════════════════════╝
    

@coolreader18 coolreader18 force-pushed the noa/ts-benchmark-client branch from 4e62e12 to 2e927ce Compare February 27, 2026 22:59
@coolreader18 coolreader18 requested a review from Centril March 2, 2026 18:14
@Centril
Copy link
Contributor

Centril commented Mar 2, 2026

The code here looks good; I just need to run the benchmarks myself before approving.

@Centril
Copy link
Contributor

Centril commented Mar 3, 2026

This gives pretty much exact parity with the rust client on my machine.

Unfortunately, I cannot confirm this on PhoenixNAP:

With this PR:

All numbers are rounded (ceil) to the nearest whole thousand TPS.

Typescript module

  • 110k
  • 100k
  • 105k
  • 108k
  • 106k
    average = 105.8k

Rust module

  • 108k
  • 120k
  • 107k
  • 117k
  • 119k
    average = 114.2k

Without this PR

Typescript module

  • 120k
  • 120k
  • 120k
  • 120k
  • 119k
    average = 119.8k

Rust module

  • 160k
  • 162k
  • 162k
  • 162k
  • 160k
    average = 161.2k

@Centril
Copy link
Contributor

Centril commented Mar 3, 2026

(The variance is also clearly higher with a typescript client.)

Copy link
Contributor

@Centril Centril left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We decided at a meeting that the numbers were OK but that we would keep around the rust client also.

@clockwork-labs-bot clockwork-labs-bot added this pull request to the merge queue Mar 3, 2026
Merged via the queue into master with commit 0b30b16 Mar 4, 2026
34 of 35 checks passed
@coolreader18 coolreader18 deleted the noa/ts-benchmark-client branch March 4, 2026 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants