Skip to content

Add support for validation jobs to sync.ts, and refactor#3541

Open
mbg wants to merge 6 commits intomainfrom
mbg/pr-checks/validation-jobs
Open

Add support for validation jobs to sync.ts, and refactor#3541
mbg wants to merge 6 commits intomainfrom
mbg/pr-checks/validation-jobs

Conversation

@mbg
Copy link
Member

@mbg mbg commented Mar 4, 2026

The primary goal of this PR is to add support for additional, validation jobs to sync.ts. My motivation for this is #3519 (review). To add a PR check which does that, the most sensible way of testing it would be to run a second job after the main one which verifies that the artifact was successfully uploaded.

This PR doesn't yet add or modify a PR check which makes use of this new functionality. I will add that as part of #3519 once this PR is merged.

I have refactored a few things in the first commits to break up the horrible long main function that we had initially ported from the earlier sync.py script as-is in #3526.

The refactoring has introduced some minor changes to the generated workflow files because the order of the various setup-* steps is now alphabetical, rather than effectively random. This should not have any impact on the behaviour of the workflows.

Notes for reviewers

Best reviewed commit-by-commit. Since these changes do not affect the CodeQL Action itself, a relatively lightweight review which focuses on the following is acceptable:

  • The high-level design of this new functionality. I.e. are we happy to support "validation jobs" in this way?
  • Are there any issues arising from the slight re-ordering of the setup steps / inputs in the workflows?
  • Any stylistic issues with the changes in this PR?

I have locally verified that the changes here broadly work for what I have in mind for the new workflow for #3526. Any small adjustments that may be required to support the full workflow could be added in that PR.

Risk assessment

For internal use only. Please select the risk level of this change:

  • Low risk: Changes are fully under feature flags, or have been fully tested and validated in pre-production environments and are highly observable, or are documentation or test only.

Which use cases does this change impact?

Environments:

  • Testing/None - This change does not impact any CodeQL workflows in production.

How did/will you validate this change?

  • End-to-end tests - I am depending on PR checks (i.e. tests in pr-checks).

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

  • Development/testing only - This change cannot cause any failures in production.

How will you know if something goes wrong after this change is released?

N/A

Are there any special considerations for merging or releasing this change?

  • No special considerations - This change can be merged at any time.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Consider adding a changelog entry for this change.
  • Confirm the readme and docs have been updated if necessary.

@mbg mbg self-assigned this Mar 4, 2026
@mbg mbg requested a review from a team as a code owner March 4, 2026 11:43
Copilot AI review requested due to automatic review settings March 4, 2026 11:43
@github-actions github-actions bot added the size/L May be hard to review label Mar 4, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors pr-checks/sync.ts to make workflow generation more modular, and introduces first-class support for generating additional “validation jobs” that run after the primary PR-check job.

Changes:

  • Refactor workflow generation into helpers (matrix generation, shared setup-step generation, job generation).
  • Add validationJobs support in check specifications to generate dependent post-check jobs.
  • Reorder generated workflow setup steps/inputs (notably alphabetizing setup steps), causing widespread regenerated workflow diffs.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
pr-checks/sync.ts Adds validation-job generation and refactors setup-step/matrix/job generation into reusable helpers.
.github/workflows/__with-checkout-path.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__upload-sarif.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__upload-ref-sha-input.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__unset-environment.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__swift-custom-build.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__split-workflow.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__remote-config.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__packaging-inputs-js.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__packaging-config-js.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__packaging-config-inputs-js.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__packaging-codescanning-config-inputs-js.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__multi-language-autodetect.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__local-bundle.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__go.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__go-custom-queries.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__export-file-baseline-information.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__config-input.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__build-mode-manual.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__analyze-ref-input.yml Regenerated output reflecting reordered setup steps/inputs.
.github/workflows/__all-platform-bundle.yml Regenerated output reflecting reordered setup steps/inputs.

Comment on lines +597 to +602
const { validationJobs, validationJobInputs } = generateValidationJobs(
specDocument,
checkSpecification,
checkName,
);
const combinedInputs = { ...workflowInputs, ...validationJobInputs };
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generateValidationJobs returns { validationJobs, workflowInputs }, but main destructures { validationJobs, validationJobInputs }. As written, validationJobInputs will always be undefined, so inputs required by validation jobs won’t be forwarded to workflow_dispatch/workflow_call (and TypeScript will flag the missing property). Rename the returned property or the destructured variable consistently, and consider returning a consistent shape even when there are no validation jobs (e.g. empty objects) to avoid undefined cases.

Suggested change
const { validationJobs, validationJobInputs } = generateValidationJobs(
specDocument,
checkSpecification,
checkName,
);
const combinedInputs = { ...workflowInputs, ...validationJobInputs };
const { validationJobs, workflowInputs: validationJobInputs } =
generateValidationJobs(specDocument, checkSpecification, checkName);
const combinedInputs = {
...(workflowInputs ?? {}),
...(validationJobInputs ?? {}),
};

Copilot uses AI. Check for mistakes.
*
* @returns An object containing setup steps and additional input specifications.
*/
function getSetupSteps(checkSpecification: Specification): {
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getSetupSteps is typed to accept Specification, but it’s also called with a JobSpecification from generateValidationJob. This is a TypeScript type error and makes it harder to reuse the helper for validation jobs. Change the parameter type to JobSpecification (or a shared narrower interface) since the function only relies on install* flags.

Suggested change
function getSetupSteps(checkSpecification: Specification): {
interface SetupSpecification {
installYq?: boolean;
[key: string]: unknown;
}
function getSetupSteps(checkSpecification: SetupSpecification): {

Copilot uses AI. Check for mistakes.

if (
setupSpec === undefined ||
checkSpecification[setupSpec.specProperty] === undefined
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The install-flag check in getSetupSteps only skips setup when the flag is undefined. If a spec explicitly sets an install flag to false, the steps will still be added. Use a truthy check (or !== true) so setup steps are only included when the corresponding install* flag is enabled.

Suggested change
checkSpecification[setupSpec.specProperty] === undefined
checkSpecification[setupSpec.specProperty] !== true

Copilot uses AI. Check for mistakes.
steps: [
{
name: "Install Python",
if: "matrix.version != 'nightly-latest'",
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Python setup step uses if: matrix.version != 'nightly-latest'. Validation jobs don’t define a matrix, so if a validation job enables installPython, this expression will reference an undefined matrix context and can fail evaluation. Consider generating a different Python setup step for validation jobs (no matrix-based if), or otherwise guarding the condition so it only applies when a matrix exists.

Suggested change
if: "matrix.version != 'nightly-latest'",

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L May be hard to review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants