accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dylan Hutchison <dhutc...@cs.washington.edu>
Subject Re: [DISCUSS] Moving away from Thrift
Date Fri, 17 Nov 2017 07:08:25 GMT
I don't know if it is mature yet, but Apache Arrow has an RPC format and
bindings for several languages.  See slide 37+ here:

https://www.slideshare.net/julienledem/the-columnar-roadmap-apache-parquet-and-apache-arrow


Arrow's RPC would be particularly advantageous if the Arrow format were
used throughout Accumulo, but perhaps there is some benefit with just using
Arrow's RPC.

On Thu, Nov 16, 2017 at 9:49 PM, Christopher <ctubbsii@apache.org> wrote:

> On Fri, Nov 17, 2017 at 12:26 AM Mike Drob <mdrob@apache.org> wrote:
>
> > Apache projects are highly discouraged from hostile forks of other ASF
> > projects. I'm not sure if there's a written foundation policy for it or
> if
> > it's simply a gentleman's agreement, but I've seen it come up in other
> > places and it has never been pretty.
> >
> > Which alternatives are you considering, Christopher? Avro RPC keeps us in
> > the Apache umbrella. I've seen Accumulo+Kryo work, but it always felt
> > kludgy. Protobuf has it's own issues.
> >
> >
> I don't really have enough information about any to consider at this point.
> The main purpose of this thread is to solicit suggestions for alternatives
> to consider. I'm not familiar with Avro RPC. As I understand it, Avro is
> serialization (same with Kryo and Protobuf). I think we have tons of
> options for serialization, but the main thing Thrift does for us that I
> haven't really seen elsewhere is the RPC language definition and
> boilerplate network/protocol stuff.
>
>
> > There may just be no good RPC solution existing today.
> >
> >
> Yeah. That is a concern. :/
>
>
> > Mike
> >
> > On Thu, Nov 16, 2017, 10:58 PM Christopher <ctubbsii@apache.org> wrote:
> >
> > > The current Thrift issue has already been fixed with a patch. Their PMC
> > > needs to release it, though.
> > >
> > > Following ASF's commitment to "community over code", I think it would
> be
> > > inappropriate for an Apache project to fork another active project
> while
> > > that community still exists. It's better to work with them if we can,
> and
> > > to use another dependency if we can't. There may be ASF policy against
> > such
> > > forking, but that may only apply to forking non-ASF projects. In any
> > case,
> > > I don't think it's a good idea.
> > >
> > > Also, even if we are able to resolve the current issue of releasing a
> > > version without the spammy print statement, I think there's value in
> > > discussing possible alternatives and their pros/cons. There's no
> timeline
> > > for this. Consider this an open-ended discussion regarding RPC
> > > alternatives. I just want to gather those alternatives into one place
> to
> > > discuss.
> > >
> > >
> > > On Thu, Nov 16, 2017 at 11:43 PM Ed Coleman <dev1@etcoleman.com>
> wrote:
> > >
> > > > Have we tried fixing the current issue and then submitting a
> > > pull-request?
> > > >
> > > > I'd favor first submitting a pull request and any other help that we
> > can
> > > > provide to get it adopted and released soon - failing that we could
> > fork
> > > > the project and go from there. That could offer us a path to correct
> > the
> > > > immediate issue and offer time to consider other alternatives.
> > > >
> > > > Ed Coleman
> > > >
> > > > -----Original Message-----
> > > > From: Christopher [mailto:ctubbsii@apache.org]
> > > > Sent: Thursday, November 16, 2017 11:36 PM
> > > > To: accumulo-dev <dev@accumulo.apache.org>
> > > > Subject: [DISCUSS] Moving away from Thrift
> > > >
> > > > Accumulo Devs,
> > > >
> > > > I think it's time we start seriously thinking about moving away from
> > > > Thrift and considering alternatives.
> > > > For me, https://issues.apache.org/jira/browse/THRIFT-4062 is
> becoming
> > > the
> > > > last straw.
> > > >
> > > > Thrift is a neat idea, but to be blunt: there seems to be a
> fundamental
> > > > lack of care or interest from the Thrift developers at the current
> > > moment.
> > > >
> > > > Some of the problems we've seen over the years: Every version is
> > > > fundamentally incompatible with other versions. Repeated
> flip-flopping
> > > > regressions seems to occur with each release. Fundamental design
> > concepts
> > > > like distinguishing server-side exceptions (TApplicationException vs.
> > > > TException) are undermined without consideration of the initial
> design.
> > > > And now, a serious bug (a spammy debugging print statement) was left
> in
> > > for
> > > > nearly a year now (still exists in current version), and no response
> > from
> > > > the PMC to indicate any willingness to release a fix. Repeated
> requests
> > > to
> > > > the developer list has gone ignored. And, I'm not even counting my
> > > requests
> > > > for assistance debugging a compiler issue on s390x arch having also
> > gone
> > > > ignored.
> > > >
> > > > These problems are not exclusive to Accumulo. Many of these are
> > problems
> > > > that Cassandra has also faced, and I'm sure there are others.
> > > >
> > > > It's possible that Thrift can remedy the situation. None of these
> > > problems
> > > > are insurmountable, and none of them are beyond fixes, particularly
> if
> > we
> > > > can afford to volunteer more to help out. My intention is not to
> throw
> > a
> > > > fellow Apache project under the bus, and I do not intend to give up
> > > > reporting bugs, and contributing patches to Thrift where appropriate.
> > > But,
> > > > I think we also need to think realistically, and consider
> alternatives,
> > > if
> > > > Thrift development does not go in a direction which is favorable to
> > > > Accumulo.
> > > >
> > > > So, with that in mind, any suggestions for alternatives? With
> > pros/cons?
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message