hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Radim Kolar <...@filez.com>
Subject Re: profiling hdfs write path
Date Mon, 26 Nov 2012 03:07:23 GMT

> Currently it's CPU-intensive for several reasons:
> 1) It doesn't yet use the native CRC code
> 2) It makes several unnecessary copies and byte buffer allocations, both in
> the client and in the DataNode
> There are open JIRAs for these, and I have a preliminary patch which helped
> a lot, but it hasn't been high priority.
can you attach crc path there? 
i will finish it.

View raw message