hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5074) support checksums in HBase block cache
Date Thu, 23 Feb 2012 22:51:52 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215153#comment-13215153
] 

Phabricator commented on HBASE-5074:
------------------------------------

mbautin has commented on the revision "[jira] [HBASE-5074] Support checksums in HBase block
cache".

  @dhruba: some more comments inline.

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:451-452 Assign headerSize()
to a local variable instead of calling it twice.
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:529-530 Call headerSize()
once and store in a local variable.
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1232 do do -> do
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1642-1644 Store and reuse
part of the previous error message.
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1636 Check if WARN level
messages are enabled and only generate the message string in that case.
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1848 double semicolon (does
not matter)
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java:424 What if istream !=
istreamNoFsChecksum but istreamNoFsChecksum == null?
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:3610-3612 Not sure how this
is related to HBase-level checksum checking
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:265 Make this conf key a constant
in HConstants
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:275 conf key -> HConstants
  src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java:40-43 This is unnecessary because
the default toString would do the same.
  src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java:57-60 This is unnecessary because
the default toString would do the same.
  src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java:103-106 This is unnecessary
because the default toString would do the same.
  src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java:143-144 It looks like toString
would to this.
  src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java:179 Would not the built-in
enum method valueOf do what this function is doing?
  src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockCompatibility.java:1 This file
still seems to contain a lot of copy-and-paste from TestHFileBlock. Are you planning to address
that?

REVISION DETAIL
  https://reviews.facebook.net/D1521

                
> support checksums in HBase block cache
> --------------------------------------
>
>                 Key: HBASE-5074
>                 URL: https://issues.apache.org/jira/browse/HBASE-5074
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: D1521.1.patch, D1521.1.patch, D1521.2.patch, D1521.2.patch, D1521.3.patch,
D1521.3.patch, D1521.4.patch, D1521.4.patch, D1521.5.patch, D1521.5.patch, D1521.6.patch,
D1521.6.patch, D1521.7.patch, D1521.7.patch, D1521.8.patch, D1521.8.patch, D1521.9.patch,
D1521.9.patch
>
>
> The current implementation of HDFS stores the data in one block file and the metadata(checksum)
in another block file. This means that every read into the HBase block cache actually consumes
two disk iops, one to the datafile and one to the checksum file. This is a major problem for
scaling HBase, because HBase is usually bottlenecked on the number of random disk iops that
the storage-hardware offers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message