arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <wesmck...@gmail.com>
Subject Re: [Discuss][Format] Arrow Flight URI scheme proposal
Date Mon, 01 Apr 2019 17:37:18 GMT
I would like to propose a vote on this feature this week. I'll wait
another day or so before starting the vote

On Fri, Mar 29, 2019 at 4:14 PM David Li <li.davidm96@gmail.com> wrote:
>
> Hey Wes,
>
> Thanks for the comments. I've updated the doc a bit to reflect some
> options for supporting "hybrid" transports.
>
> Best,
> David
>
> On 3/29/19, Wes McKinney <wesmckinn@gmail.com> wrote:
> > hi David,
> >
> > This seems like a reasonable evolution from where we are now. I will
> > defer to others to comment on the low-level details
> >
> > This is sort of scope and kind of a can of worms, but one area where
> > we should invest some thought is alternative FlightData transports,
> > while allowing the "command layer" to continue to be gRPC. One such
> > possible alternative scheme includes:
> >
> > * gRPC-over-TCP commands (actions, etc.)
> > * Movement of IPC messages using RDMA (I have never actually used RDMA
> > but it has been brought up to me as a topic of interest by multiple
> > parties now)
> >
> > If a server supports an alternative protocols (e.g. gRPC-based for
> > compatibility, plus RDMA for clients that implement it) then perhaps
> > this information can be encoded in URIs. I'm not sure if there's prior
> > art to look at on this design-wise
> >
> > - Wes
> >
> > On Wed, Mar 27, 2019 at 1:24 PM David Li <li.davidm96@gmail.com> wrote:
> >>
> >> Hi all,
> >>
> >> We'd like to propose a URI scheme for Flight, in anticipation of
> >> supporting multiple transports, and different configurations of the
> >> gRPC transport. This will change Flight.proto[1] in format/ in a
> >> backwards-incompatible way. This aims to fix ARROW-4651[2].
> >>
> >> The proposal can be found here (it should be commentable by all):
> >> https://docs.google.com/document/d/1Eps9eHvBc_qM8nRsTVwVCuWwHoEtQ-a-8Lv5dswuQoM/edit
> >>
> >> Any and all feedback is appreciated!
> >>
> >> A draft PR is up for Java/C++/Python, though it is far from complete:
> >> https://github.com/apache/arrow/pull/4047
> >>
> >> [1]: https://github.com/apache/arrow/blob/master/format/Flight.proto
> >> [2]: https://issues.apache.org/jira/browse/ARROW-4651
> >>
> >> Best,
> >> David
> >

Mime
View raw message