hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7446) Implement CRC32C native code using SSE4.2 instructions
Date Mon, 22 Aug 2011 21:36:29 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089036#comment-13089036
] 

Todd Lipcon commented on HADOOP-7446:
-------------------------------------

Hey Kihwal. I tried your pipelined CRC implementation with HBase and it's causing lots of
checksum errors (no segfaults, just an incorrect mismatched checksum). I haven't had a chance
to debug it yet, but given the issues I think I'll move forward committing the non-pipelined
implementation and we can do the pipelined one in a follow-on JIRA?

> Implement CRC32C native code using SSE4.2 instructions
> ------------------------------------------------------
>
>                 Key: HADOOP-7446
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7446
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: native
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.23.0
>
>         Attachments: hadoop-7446.txt, hadoop-7446.txt, hadoop-7446.txt, pipelined_on_todds_patch.txt,
pipelined_with_todds_patch.txt
>
>
> Once HADOOP-7445 is implemented, we can get further performance improvements by implementing
CRC32C using the hardware support available in SSE4.2. This support should be dynamically
enabled based on CPU feature flags, and of course should be ifdeffed properly so that it doesn't
break the build on architectures/platforms where it's not available.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message