GH-43854: [C++] Expose the set of device types where a ChunkedArray is allocated #43853
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
And move `enum DeviceAllocationType` to its own lightweight header.
github-actions
bot
added
Component: C++
awaiting committer review
felipecrv
changed the title
|
|
felipecrv
commented
Aug 27, 2024
github-actions
bot
added
awaiting changes
bkietz
reviewed
Aug 27, 2024
github-actions
bot
added
awaiting change review
github-actions
bot
added
awaiting changes
bkietz
approved these changes
Aug 28, 2024
github-actions
bot
added
awaiting merge
|
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 0bc91dd. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them. |
mapleFU
pushed a commit
to mapleFU/arrow
that referenced
this pull request
Sep 3, 2024
…rray is allocated (apache#43853) ### Rationale for this change `ChunkedArray`s allow flexible allocation of arrays -- the whole array doesn't have to be allocated in huge contiguous buffers. Nothing today prevents chunked arrays from being made of chunks allocated in different devices and that is good. But we need a way to query the set of devices where a chunked array is allocated at. This PR adds that missing part. ### What changes are included in this PR? Addition of: - the `DeviceAllocationTypeSet` class - `ChunkedArray::device_types()` - `Datum::device_types()` Moved `enum DeviceAllocationType` to the `type_fwd.h` header because `device.h` is too expensive of a header to hold this widely used `enum`. ### Are these changes tested? Added more asserts to `chunked_array_test.cc`. ### Are there any user-facing changes? New APIs. * GitHub Issue: apache#43854 Authored-by: Felipe Oliveira Carvalho <felipekde@gmail.com> Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
khwilson
pushed a commit
to khwilson/arrow
that referenced
this pull request
Sep 14, 2024
…rray is allocated (apache#43853) ### Rationale for this change `ChunkedArray`s allow flexible allocation of arrays -- the whole array doesn't have to be allocated in huge contiguous buffers. Nothing today prevents chunked arrays from being made of chunks allocated in different devices and that is good. But we need a way to query the set of devices where a chunked array is allocated at. This PR adds that missing part. ### What changes are included in this PR? Addition of: - the `DeviceAllocationTypeSet` class - `ChunkedArray::device_types()` - `Datum::device_types()` Moved `enum DeviceAllocationType` to the `type_fwd.h` header because `device.h` is too expensive of a header to hold this widely used `enum`. ### Are these changes tested? Added more asserts to `chunked_array_test.cc`. ### Are there any user-facing changes? New APIs. * GitHub Issue: apache#43854 Authored-by: Felipe Oliveira Carvalho <felipekde@gmail.com> Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
ChunkedArrays allow flexible allocation of arrays -- the whole array doesn't have to be allocated in huge contiguous buffers. Nothing today prevents chunked arrays from being made of chunks allocated in different devices and that is good. But we need a way to query the set of devices where a chunked array is allocated at. This PR adds that missing part.What changes are included in this PR?
Addition of:
DeviceAllocationTypeSetclassChunkedArray::device_types()Datum::device_types()Moved
enum DeviceAllocationTypeto thetype_fwd.hheader becausedevice.his too expensive of a header to hold this widely usedenum.Are these changes tested?
Added more asserts to
chunked_array_test.cc.Are there any user-facing changes?
New APIs.