Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D1E33200B99 for ; Wed, 5 Oct 2016 10:20:22 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id D0B49160AEA; Wed, 5 Oct 2016 08:20:22 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EA7BF160ADE for ; Wed, 5 Oct 2016 10:20:21 +0200 (CEST) Received: (qmail 64627 invoked by uid 500); 5 Oct 2016 08:20:21 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 64353 invoked by uid 99); 5 Oct 2016 08:20:21 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Oct 2016 08:20:21 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id B44D02C2A67 for ; Wed, 5 Oct 2016 08:20:20 +0000 (UTC) Date: Wed, 5 Oct 2016 08:20:20 +0000 (UTC) From: "Hudson (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-16644) Errors when reading legit HFile Trailer of old (v2.0) format file MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 05 Oct 2016 08:20:23 -0000 [ https://issues.apache.org/jira/browse/HBASE-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548042#comment-15548042 ] Hudson commented on HBASE-16644: -------------------------------- FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #30 (See [https://builds.apache.org/job/HBase-1.3-JDK7/30/]) HBASE-16644 Errors when reading legit HFile Trailer of old (v2.0) format (antonov: rev 737069c037cb8618fe2db5f1f5872d846a17fb01) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java > Errors when reading legit HFile Trailer of old (v2.0) format file > ----------------------------------------------------------------- > > Key: HBASE-16644 > URL: https://issues.apache.org/jira/browse/HBASE-16644 > Project: HBase > Issue Type: Bug > Components: HFile > Affects Versions: 1.3.0, 1.4.0 > Reporter: Mikhail Antonov > Assignee: Mikhail Antonov > Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16644.branch-1.3.patch, HBASE-16644.branch-1.3.patch > > > There seems to be a regression in branch 1.3 where we can't read HFile trailer(getting "CorruptHFileException: Problem reading HFile Trailer") on some HFiles that could be successfully read on 1.2. > I've seen this error manifesting in two ways so far. > {code}Caused by: org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile Trailer from file > at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497) > at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525) > at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1164) > at org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:259) > at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:427) > at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:528) > at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:518) > at org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:652) > at org.apache.hadoop.hbase.regionserver.HStore.access$000(HStore.java:117) > at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:519) > at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:516) > ... 6 more > Caused by: java.io.IOException: Invalid HFile block magic: \x00\x00\x04\x00\x00\x00\x00\x00 > at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:155) > at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:167) > at org.apache.hadoop.hbase.io.hfile.HFileBlock.(HFileBlock.java:344) > at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1735) > at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1558) > at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlock(HFileBlock.java:1397) > at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlockWithBlockType(HFileBlock.java:1405) > at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.(HFileReaderV2.java:156) > at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:485) > {code} > and second > {code} > Caused by: org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile Trailer from file > at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497) > at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525) > at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1164) > at org.apache.hadoop.hbase.io.HalfStoreFileReader.(HalfStoreFileReader.java:104) > at org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:256) > at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:427) > at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:528) > at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:518) > at org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:652) > at org.apache.hadoop.hbase.regionserver.HStore.access$000(HStore.java:117) > at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:519) > at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:516) > ... 6 more > Caused by: java.io.IOException: Premature EOF from inputStream (read returned -1, was trying to read 10083 necessary bytes and 24 extra bytes, successfully read 1072 > at org.apache.hadoop.hbase.io.hfile.HFileBlock.readWithExtra(HFileBlock.java:737) > at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1459) > at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1712) > at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1558) > at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlock(HFileBlock.java:1397) > at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlockWithBlockType(HFileBlock.java:1405) > at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.(HFileReaderV2.java:156) > at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:485) > {code} > In my case this problem was reproducible by running `hbase hfile -m -f` command. There seem to be two changes in behavior introduced by HBASE-15477 (cc [~stack]). > One is that HFileBlock#getOnDiskSizeWithHeader always assumes that checksum verification is true (this behavior results in onDiskSizeWithHeader being calculated differently in 1.2 and 1.3 for some cases). > Second is that in HFileBlock constructor we always attempt to retrive checksum-related fields from the header even if no checksum verification is going on. > Attached is the patch which when applied allows 1.3 to read again the same HFiles. -- This message was sent by Atlassian JIRA (v6.3.4#6332)