hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8323) Low hanging checksum improvements
Date Wed, 06 Nov 2013 17:53:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815068#comment-13815068

Todd Lipcon commented on HBASE-8323:

You probably want to use it via DataChecksum, which is already a public class. It has the
right logic to fallback to the Java implementation if the native one isn't available.

> Low hanging checksum improvements
> ---------------------------------
>                 Key: HBASE-8323
>                 URL: https://issues.apache.org/jira/browse/HBASE-8323
>             Project: HBase
>          Issue Type: Improvement
>          Components: Performance
>            Reporter: Enis Soztutar
> Over at Hadoop land, [~tlipcon] had done some improvements for checksums, a native implementation
for CRC32C (HADOOP-7445) and bulk verify of checksums (HADOOP-7444). 
> In HBase, we can do
>  - Also develop a bulk verify API. Regardless of hbase.hstore.bytes.per.checksum we always
want to verify of the whole checksum for the hfile block.
>  - Enable NativeCrc32 to be used as a checksum algo. It is not clear how much gain we
can expect over pure java CRC32. 
> Though, longer term we should focus on convincing hdfs guys for inline checksums (HDFS-2699)

This message was sent by Atlassian JIRA

View raw message