arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <wesmck...@gmail.com>
Subject Re: Modifying Arrow record batch
Date Tue, 17 Nov 2020 15:11:59 GMT
There are certain narrow cases where in-place modification of an IPC
message may be possible, but in general new messages would have to be
produced

On Tue, Nov 17, 2020 at 3:06 AM Saloni Udani <saloniudani.tech@gmail.com> wrote:
>
> Hello,
> We have a use case where we use Arrow format for data transfer between different data
enrichment processors. So one processor prepares an Arrow batch which is read and enriched
by the other. Now as Arrays are immutable in Arrow what is the suggested way to modify an
Arrow record batch (adding new column/adding new values to existing column/updating value
of an existing column)? Is there a way to do this with an existing Arrow record batch or it
has to be rewritten from scratch every time?
>
>
> Regards
> Saloni Udani

Mime
View raw message