Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5B5CD10D5E for ; Fri, 14 Mar 2014 23:56:53 +0000 (UTC) Received: (qmail 44118 invoked by uid 500); 14 Mar 2014 23:56:48 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 44055 invoked by uid 500); 14 Mar 2014 23:56:47 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 43805 invoked by uid 99); 14 Mar 2014 23:56:45 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Mar 2014 23:56:45 +0000 Date: Fri, 14 Mar 2014 23:56:45 +0000 (UTC) From: "Hudson (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-10751) TestHRegion testWritesWhileScanning occasional fail since HBASE-10514 went in MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935838#comment-13935838 ] Hudson commented on HBASE-10751: -------------------------------- FAILURE: Integrated in HBase-TRUNK #5012 (See [https://builds.apache.org/job/HBase-TRUNK/5012/]) HBASE-10751 TestHRegion testWritesWhileScanning occasional fail since HBASE-10514 went in (stack: rev 1577664) * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java > TestHRegion testWritesWhileScanning occasional fail since HBASE-10514 went in > ----------------------------------------------------------------------------- > > Key: HBASE-10751 > URL: https://issues.apache.org/jira/browse/HBASE-10751 > Project: HBase > Issue Type: Bug > Reporter: stack > Assignee: stack > Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18 > > Attachments: 10751.txt > > > I saw this here https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/213/testReport/junit/org.apache.hadoop.hbase.regionserver/TestHRegion/testWritesWhileScanning/ > This patch looks to have exposed a problem in our HStore commit logic. We are supposed to crash out if we fail to write but we keep going here. I am having trouble figuring why. Let me write a little test: > {code} > 2014-03-14 01:58:48,647 DEBUG [Thread-3] regionserver.HRegionFileSystem(339): Committing store file /home/jenkins/jenkins-slave/workspace/HBase-0.98-on-Hadoop-1.1/0.98-hadoop1.1/hbase-server/target/test-data/f7999012-e166-4619-ab3c-5014e0f65007/data/default/testWritesWhileScanning/306ea000673d780f06daf2469e7f9bab/.tmp/a0e6579af25f463ebb7eebe3c043b8a0 as /home/jenkins/jenkins-slave/workspace/HBase-0.98-on-Hadoop-1.1/0.98-hadoop1.1/hbase-server/target/test-data/f7999012-e166-4619-ab3c-5014e0f65007/data/default/testWritesWhileScanning/306ea000673d780f06daf2469e7f9bab/family7/a0e6579af25f463ebb7eebe3c043b8a0 > 2014-03-14 01:58:48,647 INFO [Thread-2] regionserver.HRegion(5779): writing data to region testWritesWhileScanning,,1394762315120.306ea000673d780f06daf2469e7f9bab. with WAL disabled. Data may be lost in the event of a crash. > 2014-03-14 01:58:48,648 ERROR [Thread-3] regionserver.HStore$StoreFlusherImpl(1964): Failed to commit store file /home/jenkins/jenkins-slave/workspace/HBase-0.98-on-Hadoop-1.1/0.98-hadoop1.1/hbase-server/target/test-data/f7999012-e166-4619-ab3c-5014e0f65007/data/default/testWritesWhileScanning/306ea000673d780f06daf2469e7f9bab/.tmp/a0e6579af25f463ebb7eebe3c043b8a0 > org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile Trailer from file file:/home/jenkins/jenkins-slave/workspace/HBase-0.98-on-Hadoop-1.1/0.98-hadoop1.1/hbase-server/target/test-data/f7999012-e166-4619-ab3c-5014e0f65007/data/default/testWritesWhileScanning/306ea000673d780f06daf2469e7f9bab/family7/a0e6579af25f463ebb7eebe3c043b8a0 > at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:552) > at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:580) > at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1019) > at org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:211) > at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:350) > at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:445) > at org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:551) > at org.apache.hadoop.hbase.regionserver.HStore.commitFile(HStore.java:842) > at org.apache.hadoop.hbase.regionserver.HStore.access$200(HStore.java:118) > at org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:1961) > at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1706) > at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1583) > at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1498) > at org.apache.hadoop.hbase.regionserver.TestHRegion$FlushThread.run(TestHRegion.java:3034) > Caused by: java.nio.channels.ClosedByInterruptException > at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184) > at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:282) > at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.seek(RawLocalFileSystem.java:111) > at org.apache.hadoop.fs.BufferedFSInputStream.seek(BufferedFSInputStream.java:78) > at org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:37) > at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:206) > at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237) > at org.apache.hadoop.fs.FSInputChecker.fill(FSInputChecker.java:176) > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:193) > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158) > at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384) > at org.apache.hadoop.fs.FSInputChecker.seek(FSInputChecker.java:365) > at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.seek(ChecksumFileSystem.java:271) > at org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:37) > at org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:389) > at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:537) > ... 13 more > 2014-03-14 01:58:48,657 DEBUG [pool-1-thread-1] regionserver.HRegion(1037): Closing testWritesWhileScanning,,1394762315120.306ea000673d780f06daf2469e7f9bab.: disabling compactions & flushes > 2014-03-14 01:58:48,657 INFO [pool-1-thread-1] regionserver.HRegion(1045): Running close preflush of testWritesWhileScanning,,1394762315120.306ea000673d780f06daf2469e7f9bab. > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)