[MNT] Update CI/CD with local test server and dependency matrix#1697
Open
[MNT] Update CI/CD with local test server and dependency matrix#1697
Conversation
Locally, MinIO already has more parquet files than on the test server.
Note that the previously strategy didn't work anymore if the server returned a parquet file, which is the case for the new local setup.
This means it is not reliant on the evaluation engine processing the dataset. Interestingly, the database state purposely seems to keep the last task's dataset in preparation explicitly (by having processing marked as done but having to dataset_status entry).
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1697 +/- ##
==========================================
- Coverage 53.07% 52.72% -0.35%
==========================================
Files 37 37
Lines 4381 4381
==========================================
- Hits 2325 2310 -15
- Misses 2056 2071 +15 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
PGijsbers
commented
Mar 4, 2026
.github/workflows/test.yml
Outdated
| pip install "pandas==${{ matrix.pandas-version }}" | ||
|
|
||
| # scikit-learn 1.7+ requires pandas 3.x, earlier versions use pandas 2.x | ||
| if [[ "${{ matrix.scikit-learn }}" == "1.7."* ]]; then |
Collaborator
Author
There was a problem hiding this comment.
should update this to be >=1.7
Collaborator
There was a problem hiding this comment.
scikit-learn 1.8+ isn't part of the matrix anyway, so this change doesn’t affect the current setup.
that said, I think it's fine with updating the constraint to >=1.7 in the code.
Collaborator
Author
There was a problem hiding this comment.
I'm aware, but this is something that's easily missed when it is included, and then suddenly you'll get surprised by version conflicts again :)
PGijsbers
changed the title
joaquinvanschoren
approved these changes
Mar 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Originally started in #1629, this PR spins up the services within CI uses that as a local test server for the ubuntu-based tests, closing #1614.
It also updates the test matrix to make sure included dependencies are only those that have a release on PyPI, and further restriction pandas 3.x installs to only scikit-learn 1.7 and up. Older scikit-learn versions do not play well with scikit-learn 1.6 or below.