arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ARROW-249) [Format] Define GRPC IDL / wire protocol for messaging with Arrow data
Date Thu, 11 Aug 2016 22:42:20 GMT

    [ https://issues.apache.org/jira/browse/ARROW-249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418072#comment-15418072
] 

Jacques Nadeau commented on ARROW-249:
--------------------------------------

I remember some from Kudu team (maybe [~tlipcon] or [~jdcryans]?) mentioned something about
how they had to customize the GRPC for a sidecar data batch and see some mention of it here:
https://github.com/grpc/grpc-java/issues/1054. I wonder if they could share any insights.

We implemented something custom in Drill on top of Netty before GRPC was built to satisfy
the need to support off-heap sideband messages but it would be nice to have someone else manage
the maintenance of the layer.

> [Format] Define GRPC IDL / wire protocol for messaging with Arrow data
> ----------------------------------------------------------------------
>
>                 Key: ARROW-249
>                 URL: https://issues.apache.org/jira/browse/ARROW-249
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: Format
>            Reporter: Wes McKinney
>
> In addition to memory maps / shared memory, we should be able to assemble one or more
Arrow arrays in a memory block suitable for sending over the wire with GRPC (http://www.grpc.io/).
This can be similarly adapted to other messaging / RPC frameworks. 
> We can continue to use Flatbuffers for the metadata serialization. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message