hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andronidis Anastasios <andronat_...@hotmail.com>
Subject Re: MapWritable is really bad
Date Thu, 25 Sep 2014 11:18:52 GMT
Hi, I am not saying to remove kryo. Kryo is a wonderful asset to have. I just say that kryo
+ careful data structure use = the best performance.

We must make sure that kryo is running optimally though. The library has lots of configurations
and we must make sure that we use it properly.

Cheers,
Anastasis

On 25 Σεπ 2014, at 12:58 μ.μ., Chia-Hung Lin <clin4j@googlemail.com> wrote:

> Should we roll out with our own implementation? Switching to kyro look
> like will get of this issue, but if we are going to have plugable
> serialization framework (are we?) that might help those who need it.
> 
> On 24 September 2014 17:19, Edward J. Yoon <edwardyoon@apache.org> wrote:
>> Interesting ..
>> 
>> On Fri, Sep 19, 2014 at 11:52 PM, Andronidis Anastasios
>> <andronat_asf@hotmail.com> wrote:
>>> Hello,
>>> 
>>> I remember a discussion rose upon performance issues on messages and that kryo
serializer helped a lot.
>>> 
>>> Please read this: http://www.chrisstucchio.com/blog/2011/mapwritable_sometimes_a_performance_hog.html
>>> 
>>> From a custom test I did, I was sending some messages with MapWritable as a container,
Text as key and ArrayWritable (with integers inside) as a value. Hama was reporting 20MB of
traffic. When I wrote my own Map (that implements Writable interface) I reduced the amount
from 20MB to 1.5MB..
>>> 
>>> Cheers,
>>> Anastasios
>> 
>> 
>> 
>> --
>> Best Regards, Edward J. Yoon
>> CEO at DataSayer Co., Ltd.


Mime
View raw message