hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5074) support checksums in HBase block cache
Date Mon, 06 Feb 2012 20:49:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201569#comment-13201569
] 

Phabricator commented on HBASE-5074:
------------------------------------

mbautin has commented on the revision "[jira] [HBASE-5074] Support checksums in HBase block
cache".

  @dhruba; thanks for the fixes! Here are some more comments (I still have to go through the
last 25% of the new version of the patch).

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:119 Please address
this comment. The javadoc says "major" and the variable name says "minor".
  src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:49 Please correct the
misspelling.
  src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:352 I think this function
needs to be renamed to expectAtLeastMajorVersion for clarity
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:287 I think we should either
consistently use the onDiskSizeWithHeader field or get rid of it.
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java:220 Please do use a constant
instead of "0" here for the minor version.
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:3551 Long line
  src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java:60 This lazy initialization
is not thread-safe. This also applies to other enum members below. Can the meth field be initialized
on the enum constructor, or do we rely on some classes being loaded by the time this initialization
is invoked?
  src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java:63-67 Avoid repeating "org.apache.hadoop.util.PureJavaCrc32"
three times in string form
  src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java:74-75 Avoid repeating the "java.util.zip.CRC32"
string
  src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java:98-99 Avoid repeating the string
  src/test/java/org/apache/hadoop/hbase/io/hfile/TestChecksum.java:132 Fix indentation
  src/test/java/org/apache/hadoop/hbase/io/hfile/TestChecksum.java:174 Fix indentation
  src/test/java/org/apache/hadoop/hbase/io/hfile/TestChecksum.java:71 Inconsistent formatting:
"1024   +980".

REVISION DETAIL
  https://reviews.facebook.net/D1521

                
> support checksums in HBase block cache
> --------------------------------------
>
>                 Key: HBASE-5074
>                 URL: https://issues.apache.org/jira/browse/HBASE-5074
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: D1521.1.patch, D1521.1.patch, D1521.2.patch, D1521.2.patch, D1521.3.patch,
D1521.3.patch
>
>
> The current implementation of HDFS stores the data in one block file and the metadata(checksum)
in another block file. This means that every read into the HBase block cache actually consumes
two disk iops, one to the datafile and one to the checksum file. This is a major problem for
scaling HBase, because HBase is usually bottlenecked on the number of random disk iops that
the storage-hardware offers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message