arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saloni Udani <>
Subject Modifying Arrow record batch
Date Tue, 17 Nov 2020 09:06:27 GMT
We have a use case where we use Arrow format for data transfer between
different data enrichment processors. So one processor prepares an Arrow
batch which is read and enriched by the other. Now as Arrays are immutable
in Arrow what is the suggested way to modify an Arrow record batch (adding
new column/adding new values to existing column/updating value of an
existing column)? Is there a way to do this with an existing Arrow record
batch or it has to be rewritten from scratch every time?

Saloni Udani

View raw message