Skip to content

feat(e2e): switch long-running test apps from dev to build+serve#7795

Open
jacekradko wants to merge 35 commits intomainfrom
jr/integration-build-serve
Open

feat(e2e): switch long-running test apps from dev to build+serve#7795
jacekradko wants to merge 35 commits intomainfrom
jr/integration-build-serve

Conversation

@jacekradko
Copy link
Member

@jacekradko jacekradko commented Feb 6, 2026

Summary

Switch all 27 long-running integration test apps from app.dev() (JIT compilation) to app.build() + app.serve() (pre-compiled, static serving) for faster and cacheable test runs.

Changes

  • Update serve() in the Application model to support detached mode with file-based logging and HTTP polling, aligning it with dev()
  • Fix serve() return type (pid: procpid: proc.pid)
  • Read .env file and pass vars as process env to serve commands, since production servers (e.g. react-router-serve) don't auto-load .env files like Vite dev does
  • Remove the build() error stub from longRunningApplication proxy
  • Increase global setup timeout from 90s to 5 minutes to accommodate build step
  • Fix template build/serve scripts:
    • Fix srvx static path resolution for tanstack-start (--static ../client relative to entry file dir)
    • Add srvx dependency for tanstack-start production serving
    • Set NODE_ENV=production for react-router-serve
    • Remove type checking from build scripts (tsc, vue-tsc, astro check) — not needed for integration testing
    • Handle undefined PORT env in Astro and React Router configs (PORT is only set at serve time, not build time)

CI timing comparison (dev mode vs build+serve)

Baseline = average of two recent dev-mode CI runs on another branch.

Test Suite Baseline (dev) Build+Serve Delta
ap-flows 164s 161s -2s
astro 288s 205s -83s (-29%)
billing 230s 237s +8s
cache-components 30s 29s -1s
custom 212s 214s +2s
express 146s 151s +4s
generic 310s 294s -16s
handshake 146s 143s -4s
handshake:staging 147s 144s -3s
localhost 176s 171s -5s
machine 306s 314s +8s
nextjs (v15) 376s 363s -13s
nextjs (v16) 348s 316s -32s (-9%)
nuxt 152s 158s +6s
quickstart (v15) 186s 179s -6s
quickstart (v16) 165s 169s +4s
react-router 145s 149s +4s
sessions 207s 208s +1s
sessions:staging 210s 207s -3s
tanstack 160s 163s +3s
vue 144s 144s +0s
Total 4246s 4119s -127s (-3%)

Most suites are within noise. The build overhead is paid in global.setup (before test timing), while test-time page loads benefit from pre-compiled assets. The biggest wins are in frameworks with slow dev-mode compilation (astro, nextjs). Future runs will benefit further from build caching.

Test plan

  • pnpm build passes
  • All 21 integration test suites pass in CI
  • tanstack-react-start: static assets served correctly via srvx
  • react-router: .env vars available to production server
  • Verify middleware-placement.test.ts still works (non-detached serve path unchanged)

Summary by CodeRabbit

  • New Features

    • Added detached server mode with configurable server URL support
  • Bug Fixes

    • Improved port configuration handling when PORT environment variable is unset
    • Enhanced environment variable loading from .env files
  • Chores

    • Simplified build scripts by removing unnecessary pre-build checks
    • Enhanced logging for server startup and initialization
    • Updated test timeouts for improved stability
    • Updated package dependencies across templates

@vercel
Copy link

vercel bot commented Feb 6, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
clerk-js-sandbox Ready Ready Preview, Comment Mar 4, 2026 11:14pm

Request Review

@changeset-bot
Copy link

changeset-bot bot commented Feb 6, 2026

⚠️ No Changeset found

Latest commit: 7a448c1

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Feb 6, 2026

Open in StackBlitz

@clerk/agent-toolkit

npm i https://pkg.pr.new/@clerk/agent-toolkit@7795

@clerk/astro

npm i https://pkg.pr.new/@clerk/astro@7795

@clerk/backend

npm i https://pkg.pr.new/@clerk/backend@7795

@clerk/chrome-extension

npm i https://pkg.pr.new/@clerk/chrome-extension@7795

@clerk/clerk-js

npm i https://pkg.pr.new/@clerk/clerk-js@7795

@clerk/dev-cli

npm i https://pkg.pr.new/@clerk/dev-cli@7795

@clerk/expo

npm i https://pkg.pr.new/@clerk/expo@7795

@clerk/expo-passkeys

npm i https://pkg.pr.new/@clerk/expo-passkeys@7795

@clerk/express

npm i https://pkg.pr.new/@clerk/express@7795

@clerk/fastify

npm i https://pkg.pr.new/@clerk/fastify@7795

@clerk/hono

npm i https://pkg.pr.new/@clerk/hono@7795

@clerk/localizations

npm i https://pkg.pr.new/@clerk/localizations@7795

@clerk/nextjs

npm i https://pkg.pr.new/@clerk/nextjs@7795

@clerk/nuxt

npm i https://pkg.pr.new/@clerk/nuxt@7795

@clerk/react

npm i https://pkg.pr.new/@clerk/react@7795

@clerk/react-router

npm i https://pkg.pr.new/@clerk/react-router@7795

@clerk/shared

npm i https://pkg.pr.new/@clerk/shared@7795

@clerk/tanstack-react-start

npm i https://pkg.pr.new/@clerk/tanstack-react-start@7795

@clerk/testing

npm i https://pkg.pr.new/@clerk/testing@7795

@clerk/ui

npm i https://pkg.pr.new/@clerk/ui@7795

@clerk/upgrade

npm i https://pkg.pr.new/@clerk/upgrade@7795

@clerk/vue

npm i https://pkg.pr.new/@clerk/vue@7795

commit: 7a448c1

@jacekradko jacekradko changed the title feat(integration): switch long-running test apps from dev to build+serve feat(e2e): switch long-running test apps from dev to build+serve Feb 6, 2026
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 6, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 6645c8d9-8664-4d14-944b-d47375a44d55

📥 Commits

Reviewing files that changed from the base of the PR and between 978e231 and 7a448c1.

📒 Files selected for processing (1)
  • integration/templates/tanstack-react-start/package.json

📝 Walkthrough

Walkthrough

Added exported helper resolveServerUrl(optsServerUrl, fallbackServerUrl, port) and unit tests. Extended serve options to include detached and serverUrl, compute a runtime server URL, support manualStart early return, spawn detached servers with stdout/stderr redirection and readiness polling, set state.serverUrl, and return PID. Serve now loads a .env file into the environment. longRunningApplication logs more phases, applies env before setup, runs a 120s build then serve({ detached: true }), and exposes build() as a resolved Promise. Playwright timeout increased to 300000 ms. Several templates adjust server.port, simplify build scripts, and add an Astro node adapter.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(e2e): switch long-running test apps from dev to build+serve' accurately and concisely describes the main change: transitioning integration test apps from dev mode (JIT compilation) to build+serve mode (pre-compiled, static serving).
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@integration/models/longRunningApplication.ts`:
- Around line 109-110: The serve result (port, serverUrl, pid) is being
destructured into new local consts and not assigned to the outer variables, so
dev() can still see undefined; change the destructuring so the returned values
are assigned to the outer-scoped variables (e.g., remove "const" or explicitly
set port = ..., serverUrl = ..., pid = ...) before calling
stateFile.addLongRunningApp({ port, serverUrl, pid, id, appDir: app.appDir, env:
params.env.toJson() }) so the in-memory state is updated immediately after
app.serve() and dev()/init() will read the correct runtime info.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In `@integration/models/application.ts`:
- Line 1: Add unit/integration tests that exercise the build+serve detached
flow: create a new test that invokes the code path which performs buildAndServe
(or startDetachedServe) with detached logging enabled, stubbing/mocking execSync
and any process/spawn calls to simulate detached child processes, and assert
that readinessCheck (or the readiness polling function) is called and that
detached logging output is captured/forwarded as expected; ensure the test fails
on missing readiness and covers both successful and timeout/error cases so the
change is covered by CI.
- Around line 150-156: The getServerUrl helper mis-detects presence of a port by
using .includes(':'') (which sees the scheme) so a value like
opts.serverUrl="http://localhost" won’t get the runtime port appended and the
process PORT and runtime URL can become unsynchronized; update getServerUrl to
parse opts.serverUrl with the URL constructor (or equivalent) to check for an
explicit port and if none append `:${port}` to the host, and also ensure
process.env.PORT (or the runtime PORT variable) is set to String(port) so
serve()/waitForServer use the same port; additionally add unit/integration tests
exercising serve() with a custom opts.serverUrl without port and the detached
mode flow to verify the returned server URL and readiness checks are correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant