parquet bench + cuda changes to scan clickbench#6739
parquet bench + cuda changes to scan clickbench#6739onursatici wants to merge 14 commits intoos/gpu-scan-benchfrom
Conversation
Signed-off-by: Onur Satici <onur@spiraldb.com>
Signed-off-by: Onur Satici <onur@spiraldb.com>
Signed-off-by: Onur Satici <onur@spiraldb.com>
Signed-off-by: Onur Satici <onur@spiraldb.com>
Signed-off-by: Onur Satici <onur@spiraldb.com>
Signed-off-by: Onur Satici <onur@spiraldb.com>
Signed-off-by: Onur Satici <onur@spiraldb.com>
| # This serves as the baseline for comparing against Vortex GPU scans. | ||
| # | ||
| # Usage: | ||
| # uv run bench_parquet.py dataset.parquet --iterations 5 |
There was a problem hiding this comment.
this is a standalone uv script to do the same scan we do on gpu-scan-bench above, but for parquet instead of vortex
| materialize_constant_decimal::<D>(array, decimal_dtype, validity, ctx).await | ||
| }) | ||
| } | ||
| DType::Extension(ext_dtype) => { |
There was a problem hiding this comment.
I needed this to solve a panic on datetimeparts
|
|
||
| // Components may decompress as unsigned (e.g. from BitPacked). Reinterpret | ||
| // as signed since the CUDA kernel only has signed variants and casts | ||
| // everything to int64_t anyway — the bit pattern is identical. |
There was a problem hiding this comment.
I don't know why I was getting a uint here on datetimeparts, so hacked around this
| // the same for signed/unsigned pairs (e.g. i16/u16). | ||
| if let Some(bitpacked) = array.encoded().as_opt::<BitPackedVTable>() { | ||
| match_each_integer_ptype!(bitpacked.ptype(), |P| { | ||
| match_each_integer_ptype!(array.ptype(), |P| { |
There was a problem hiding this comment.
this is an irrelevant fix that I did first, but turned out it is datetimeparts
| )); | ||
|
|
||
| #[cfg(debug_assertions)] | ||
| validate_decompress_results(&plan, device_actual_sizes, device_statuses).await?; |
There was a problem hiding this comment.
skip validation on release builds because they force a host copy
Merging this PR will degrade performance by 10.2%
Performance Changes
Comparing Footnotes
|
Signed-off-by: Onur Satici <onur@spiraldb.com>
Signed-off-by: Onur Satici <onur@spiraldb.com>
Signed-off-by: Onur Satici <onur@spiraldb.com>
Signed-off-by: Onur Satici <onur@spiraldb.com>
f08430d to
eed8627
Compare
Summary
Closes: #000
Testing