arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damien Chaillou <>
Subject Re: Example for Apache Arrow Flight
Date Wed, 13 May 2020 02:06:01 GMT
Hi Andy,
thanks for the VERY quick response and links, I will study them.
However, I had in mind an implementation like the example you implemented
in the Rust example folder[1], I mean implementing the FlightService trait
from the gRPC service. This is why meant to build a FlightData, directly
[2]. Would it mean I "just" have to transform my VectorSchemaRoot in a
ByteString (set in the dataBody of the FlightData object) that will be
streamed over gRPC?

(I wanted to use Scala AkkaStream gRPC to implement my server, because I
don't think *org.apache.arrow.flight.FlightServer* will fit my needs.)

Or Maybe I didn't understand how a Flight Server should be implemented and
I got it wrong?




Le mar. 12 mai 2020 à 21:58, Andy Grove <> a écrit :

> Hi Damien,
> Here is a brief answer that hopefully at least points you in the right
> direction.
> You need to use the VectorSchemaRoot class to build batches of data. There
> is some documentation on how to do that [1]. Then, in your FlightProducer
> implementation, you need to pass the batches to the
> FlightProducer.ServerStreamListener using the "start" and "next" methods
> when batches are ready to be sent. There is sample code in the Arrrow repo
> [2] and there is a Kotlin example that I wrote here [3].
> Andy.
> [1]
> [2]
> [3]
> On Tue, May 12, 2020 at 6:43 PM Damien Chaillou <>
> wrote:
>> Hi!
>> I'm currently playing with Apache Arrow Flight in java and cant get my
>> head around how to implement something.
>> In the *doGet* method, for example, I'm doing simple JDBC calls that I
>> would like to stream over.
>> If I understand correctly, the *FlightData*'s body should be a
>> ArrowMessage serialised as a ByteString (?) I built from a ResultSet from
>> my JDBC call. I though using JdbcToArrow
>> <>
>> class to help, but I can't find any example of how to do such thing.
>> I came with few questions :
>>    - What is the data we must set in dataBody ?
>>    - If those are ArrowMessages, how can I map a ResultSet to this type?
>>    - How do we serialise to ByteString objects typed like ArrowMessage,
>>    Schema ... ?
>> Could anyone point a piece of code/blog post/anything to me please ?
>> My toy project would be a generic proxy server in front of any database
>> with available JDBC drivers that could stream queries over Arrow Flight
>> (gRPC).
>> Cheers, thanks!
>> Damien

View raw message