You want to use the IPC mechanism to pass arrow buffers between languages
First get a buffer:
import pyarrow as pa
data = [
pa.array([1, 2, 3, 4]),
pa.array(['foo', 'bar', 'baz', None]),
pa.array([True, None, False, True])
batch = pa.record_batch(data, names=['f0', 'f1', 'f2'])
sink = pa.BufferOutputStream()
writer = pa.ipc.new_stream(sink, batch.schema)
buf = sink.getvalue()
The buffer could be written to Redis, to a file etc. For redis I think `r.set("key", buf.hex())` is easiest, you don't have to worry about encoding.
On the java side something like:
Jedis jedis = new Jedis();
String buf = jedis.get("key");
RootAllocator rootAllocator = new RootAllocator(Long.MAX_VALUE);
ByteArrayInputStream in = new ByteArrayInputStream(hexStringToByteArray(buf));
ArrowStreamReader stream = new ArrowStreamReader(in, rootAllocator);
VectorSchemaRoot vsr = stream.getVectorSchemaRoot();
And the VectorSchemaRoot holds the correct Arrow Buffer.
While Redis will work for this you might find a file or socket a bit more ergonomic in Arrow. The Plasma object store is also an option which you can think of as a primitive Redis specifically for Arrow Buffers. Finally, if you are using Redis as a message bus you might find the Arrow RPC mechanism Arrow Flight is a good choice.