arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bjoern Bachmann <>
Subject [C++] Data Movement using Apache Arrow
Date Wed, 19 May 2021 17:35:57 GMT
<html><head></head><body><div style="font-family: Verdana;font-size:
12.0px;"><div>Hi Apache Arrow User,<br/>
I&#39;m looking for some help to get started with Apache Arrow. We have a distributed
system which has the following operation sequence:<br/>
Compute Node 1: Captures measurement data, 2x float arrays (32Bit with 150k values per array)
which are transfered to a kafka cluster<br/>
Compute Node 2: Should listen on kafka for new data logs (the data logs are microbatches of
measurment data), fetch data when received and then transfer it to another process/thread
where the data is analysed.<br/>
The system is mainly written in C++ and parts of the final data analysis in c#. How could
I effectivly utilize Arrow in such a scenario? Or should I use an alternative solutions? Are
there any examples or code snippets which could help?<br/>

View raw message