hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Radim Kolar <...@filez.com>
Subject Re: profiling hdfs write path
Date Mon, 26 Nov 2012 03:07:23 GMT

> Currently it's CPU-intensive for several reasons:
> 1) It doesn't yet use the native CRC code
> 2) It makes several unnecessary copies and byte buffer allocations, both in
> the client and in the DataNode
>
> There are open JIRAs for these, and I have a preliminary patch which helped
> a lot, but it hasn't been high priority.
can you attach crc path there? 
https://issues.apache.org/jira/browse/HDFS-3528
i will finish it.

Mime
View raw message