hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6798) HDFS always read checksum form meta file
Date Tue, 18 Sep 2012 09:32:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457716#comment-13457716

Anoop Sam John commented on HBASE-6798:

So there is no effect on doing FileSystem#setVerifyChecksum(false)
We have this piece of code in HFileSystem
// If hbase checksum verification is switched on, then create a new
    // filesystem object that has cksum verification turned off.
    // We will avoid verifying checksums in the fs client, instead do it
    // inside of hbase.
    // If this is the local file system hadoop has a bug where seeks
    // do not go to the correct location if setVerifyChecksum(false) is called.
    // This manifests itself in that incorrect data is read and HFileBlocks won't be able
to read
    // their header magic numbers. See HBASE-5885
    if (useHBaseChecksum && !(fs instanceof LocalFileSystem)) {
      this.noChecksumFs = newInstanceFileSystem(conf);
    } else {
      this.noChecksumFs = fs;

> HDFS always read checksum form meta file
> ----------------------------------------
>                 Key: HBASE-6798
>                 URL: https://issues.apache.org/jira/browse/HBASE-6798
>             Project: HBase
>          Issue Type: Bug
>          Components: performance
>    Affects Versions: 0.94.0, 0.94.1
>            Reporter: LiuLei
>            Priority: Blocker
> I use hbase0.941 and hadoop-0.20.2-cdh3u5 version.
> The HBase support checksums in HBase block cache in HBASE-5074 jira.
> The  HBase  support checksums for decrease the iops of  HDFS, so that HDFS
> dont't need to read the checksum from meta file of block file.
> But in hadoop-0.20.2-cdh3u5 version, BlockSender still read the metadata file even if
>  hbase.regionserver.checksum.verify property is ture.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message