hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3529) Use direct buffers for data in write path
Date Tue, 06 Nov 2012 20:30:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491789#comment-13491789
] 

Todd Lipcon commented on HDFS-3529:
-----------------------------------

Hey Gopal, Trevor,

I agree we need the native CRC stuff to make a substantial savings here. I don't think the
direct buffers alone are significant enough to make a noticeable difference. I had a patch
in flight a few months ago which Trevor was working off of - I'll see if I can find some time
to revive it on top of Trevor's latest work.
                
> Use direct buffers for data in write path
> -----------------------------------------
>
>                 Key: HDFS-3529
>                 URL: https://issues.apache.org/jira/browse/HDFS-3529
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node, performance
>    Affects Versions: 2.0.0-alpha
>            Reporter: Todd Lipcon
>            Assignee: Trevor Robinson
>         Attachments: dfsio-x86-trunk-vs-3529.png, HDFS-3529.patch
>
>
> The write path currently makes several unnecessary data copies in order to go to and
from byte arrays. We can improve performance by using direct byte buffers to avoid the copy.
This is also a prerequisite for native checksum calculation (HDFS-3528)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message