hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Kang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6672) BytesWritable.write(buf) use much more CPU in writeInt() then write(buf)
Date Sat, 03 Apr 2010 16:31:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853155#action_12853155

Xiao Kang commented on HADOOP-6672:

The profiling data is got from a internal version created from hadoop 0.19 trunk.

MapOutputBuffer.Buffer.write(byte[], int, int) is also not a synchronized method in the profiling
version. However, it trys to get a ReentrantLock(MapOUtputBuffer.Buffer.write(byte[], int,
int)) before witting data to buffer and this is the performance issue.

> BytesWritable.write(buf) use much more CPU in writeInt() then write(buf)
> ------------------------------------------------------------------------
>                 Key: HADOOP-6672
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6672
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io
>    Affects Versions: 0.20.2
>            Reporter: Xiao Kang
>         Attachments: BytesWritable.java.patch, screenshot-1.jpg, screenshot-2.jpg
> BytesWritable.write() use nearly 4 times of CPU in wirteInt() as write buffer. It may
be optimized.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message