flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Thrift Source: TCompactProtocol vs TBinaryProtocol
Date Fri, 18 Apr 2014 00:08:57 GMT
The unit tests use the RPC client which also uses the TCompactProtocol,
probably that is why it is failing.


On Thu, Apr 17, 2014 at 2:00 PM, Stefan Krawczyk <stefan@nextdoor.com>wrote:

> Cool.
>
> We're sending events to flume from python, and there's a c-extension
> implementation of the binary protocol which is a little faster for
> serialization, so I was wanting to try it out.
>
> Yep, I managed to recompile flume, however for some reason the server
> works, but the unit tests don't...
>
> Cheers,
>
> Stefan
>
>
> On Thu, Apr 17, 2014 at 11:13 AM, Hari Shreedharan <
> hshreedharan@cloudera.com> wrote:
>
>> Unfortunately, no - there is currently no way to use TBinaryProtocol
>> instead. Is there a specific need for using the TBinaryProtocol? There is
>> no specific reason for it to be hardcoded, just that TCompactProtocol is
>> cheaper in terms of processing and space.
>>
>> To use TBinaryProtocol, you'd have to recompile Flume (or at least create
>> a custom Thrift Source which could be a copy-paste of the Flume Thrift
>> Source, replacing the TCompact with TBinary).
>>
>>
>> Hari
>>
>>
>> On Thu, Apr 17, 2014 at 11:06 AM, Stefan Krawczyk <stefan@nextdoor.com>wrote:
>>
>>> Hi,
>>>
>>> I am wanting to try and use the TBinaryProtocol instead of the
>>> TCompactProtocol for the flume thrift source. However it seems that the
>>> choice of protocol to use on the flume side is *hardcoded *to be the
>>> TCompactProtocol.
>>>
>>> So two questions:
>>> 1) Is there any particular reason for it to be hardcoded?
>>> 2) Would I run into other difficulties if I swapped out
>>> the TCompactProtocol for the TBinaryProtocol?
>>>
>>> Cheers,
>>>
>>> Stefan
>>>
>>
>>
>

Mime
View raw message