arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Barbara BOYAJIAN <barb...@elementaryrobotics.com>
Subject [C++] [Python] How to serialize and send C++ arrow types to a Python client to deserialize?
Date Mon, 10 Aug 2020 17:32:31 GMT
Hello,

I'm currently looking to use Arrow in the following use case. I am writing
a C++ client, where I need to send serialized Arrow data to Redis, and
deserialize Arrow data that is received from Redis. I'm using boost::asio
to communicate with Redis, and am able to send/receive buffers via unix and
tcp sockets. I also need this C++ client to serialize/deserialize the data
in the same format that our existing Python client does with pyarrow, so
that serialized data sent from the C++ client can be read from the Python
client and vice versa.

I wish to be able to apply the above use case to send/receive
arrow::Tensors, arrow::Tables, and arrow::Arrays.

After a bit of reading and research, I suspect that I should be using the
arrow::py library, but was hoping to get more guidance on this.

So far, I have created a C++ arrow::Table manually, wrapped it using
arrow::py::wrap_table, and have tried to use arrow::SerializeObject(...) to
serialize it. However, my approach is not working as the memory address for
the variable that is meant to hold the serialized object is 0x0.

Thank you very much in advance for your help.

Mime
View raw message