flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Krawczyk <ste...@nextdoor.com>
Subject Re: Thrift Source: TCompactProtocol vs TBinaryProtocol
Date Thu, 17 Apr 2014 21:00:16 GMT

We're sending events to flume from python, and there's a c-extension
implementation of the binary protocol which is a little faster for
serialization, so I was wanting to try it out.

Yep, I managed to recompile flume, however for some reason the server
works, but the unit tests don't...



On Thu, Apr 17, 2014 at 11:13 AM, Hari Shreedharan <
hshreedharan@cloudera.com> wrote:

> Unfortunately, no - there is currently no way to use TBinaryProtocol
> instead. Is there a specific need for using the TBinaryProtocol? There is
> no specific reason for it to be hardcoded, just that TCompactProtocol is
> cheaper in terms of processing and space.
> To use TBinaryProtocol, you'd have to recompile Flume (or at least create
> a custom Thrift Source which could be a copy-paste of the Flume Thrift
> Source, replacing the TCompact with TBinary).
> Hari
> On Thu, Apr 17, 2014 at 11:06 AM, Stefan Krawczyk <stefan@nextdoor.com>wrote:
>> Hi,
>> I am wanting to try and use the TBinaryProtocol instead of the
>> TCompactProtocol for the flume thrift source. However it seems that the
>> choice of protocol to use on the flume side is *hardcoded *to be the
>> TCompactProtocol.
>> So two questions:
>> 1) Is there any particular reason for it to be hardcoded?
>> 2) Would I run into other difficulties if I swapped out
>> the TCompactProtocol for the TBinaryProtocol?
>> Cheers,
>> Stefan

View raw message