We have a use case where we use Arrow format for data transfer between different data enrichment processors. So one processor prepares an Arrow batch which is read and enriched by the other. Now as Arrays are immutable in Arrow what is the suggested way to modify an Arrow record batch (adding new column/adding new values to existing column/updating value of an existing column)? Is there a way to do this with an existing Arrow record batch or it has to be rewritten from scratch every time?

Saloni Udani