accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <>
Subject Re: State of our RPCs
Date Tue, 01 Dec 2015 20:19:32 GMT
Gotcha. Billie just noted in IRC that, when Accumulo has custom RPC code 
a long time ago, Keith's experiment showed Thrift 500x faster. That's 
probably how we got started.

Since then, I think it's just been a good solution that works well in 
the end (development work can be hard). The lack of an integrated 
serialization and transport system is what kept up with Thrift. I could 
only come up with gRPC that could be a true replacement, but I may be 
missing others. wrote:
> I'm not suggesting that we replace Thrift (nor am I signing up to do it), just asking
for the basis of the decision and if its time to revisit. I'm totally ok with a 'no' answer.
> ----- Original Message -----
> From: "Josh Elser"<>
> To:
> Sent: Tuesday, December 1, 2015 2:36:18 PM
> Subject: Re: State of our RPCs
> To play devil's advocate: I'm not sure if it's quite that simple. For
> example, Avro has been around since 2009, but I don't think it'd be fair
> to consider Avro circa 2009 to Avro circa 2015.
> David Medinets wrote:
>> What new protocols have been introduced since the Thrift decisions? Can
>> someone provide pros and cons for that limited set of protocols?
>> On Tue, Dec 1, 2015 at 1:02 PM,<>  wrote:
>>> What was it about Thrift that drove us to use it? Was it the bindings for
>>> multiple languages? Should this decision be revisited?
>>> ----- Original Message -----
>>> From: "Josh Elser"<>
>>> To: "dev"<>
>>> Sent: Tuesday, December 1, 2015 12:49:26 PM
>>> Subject: State of our RPCs
>>> Hi --
>>> My adventures in Thrift as a part of ACCUMULO-4065 are finally coming to
>>> a close, it seems. The briefest summary I can give is that our hack to
>>> work around an 0.9.0->0.9.1 compatibility issue ended up creating a bug
>>> in a very obtuse case (when a server answering a oneway Thrift call
>>> threw an RTE or an Error).
>>> Given some other recent chatter in the project, I'm left wondering: what
>>> next?
>>> We've long considered Thrift to be a very useful tool, but extremely
>>> scary to upgrade. I think this is just another sign of this. This leaves
>>> me asking, how do we fix this?
>>> Best as I understand it, Thrift is still a relatively active project (at
>>> least their mailing list archives shows it). My impression is that the
>>> Java library is much less-so. Most of our issues to me that they
>>> ultimately stem from incompatibilities between libthrift versions and
>>> uncaught performance regressions.
>>> Assuming that to be true, do we need to make a coordinated effort to
>>> improve the upstream libthrift code? Become a part of their community,
>>> focusing on preventing these sorts of issues from ever filtering down to
>>> us? Help them generate and follow compatibility guidelines?
>>> I feel like our strategy over the past few years has been to "avert your
>>> eyes" -- if we don't touch it, it'll hopefully be ok. Perhaps we need to
>>> try something new. Thoughts?
>>> - Josh

View raw message