Hi Simon,I don't think there is a public API for this in C++. You would have to presize a values buffer to the size expected for the compressed data, have the compressor output directly to that buffer while recording the necessary offsets. You could then construct the BinaryArray directly with these buffers (I would need to double check, but you might need to construct an intermediate ArrayData object).
Hope this helps.
On Thursday, June 18, 2020, Simon Dumke <email@example.com> wrote:
I would like build RecordBatches with (besides others) a BinaryArray column containing compressed data. when filling the BinaryArray, i would like to allow the compresseor to immediately output into the Arrow Buffer instead of allocating an output buffer and then copying the data into Arrow Buffers.
Is such an approach possible? And if so - how do I achieve this?
I'd be thankfull for any insights!
-- Simon Dumke Entwickler - CoDaC Department Operation Max Planck Institut for Plasmaphysics Wendelsteinstrasse 1 17491 Greifswald, Germany Phone: +49(0)3834 88 1215