Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3A4E49BE5 for ; Wed, 5 Oct 2011 23:22:56 +0000 (UTC) Received: (qmail 89976 invoked by uid 500); 5 Oct 2011 23:22:56 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 89944 invoked by uid 500); 5 Oct 2011 23:22:56 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 89932 invoked by uid 99); 5 Oct 2011 23:22:56 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Oct 2011 23:22:56 +0000 X-ASF-Spam-Status: No, hits=-2000.5 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Oct 2011 23:22:54 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id DE1532ABFB8 for ; Wed, 5 Oct 2011 23:22:33 +0000 (UTC) Date: Wed, 5 Oct 2011 23:22:33 +0000 (UTC) From: "jiraposter@reviews.apache.org (Commented) (JIRA)" To: issues@hbase.apache.org Message-ID: <1756354.1340.1317856953910.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Commented] (HBASE-2856) TestAcidGuarantee broken on trunk MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121583#comment-13121583 ] jiraposter@reviews.apache.org commented on HBASE-2856: ------------------------------------------------------ ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2224/#review2365 ----------------------------------------------------------- this is some good stuff. nice work amit. /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java whitespace added in this file /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java maybe add one line of javadoc to describe why we need this method (this is a method that gets used pretty widely in the code to support this change) /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java should we be pulling this stuff from KeyValue constants instead of Bytes directly? /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java debug? /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java maybe move this Bytes.SIZEOF_LONG into some other constant that reflects what this is for? KeyValue.MEMSTORE_TS_SIZE or some such thing /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java comment here /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java accidental new line w/ whitespace /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java introducing whitespace here and throughout file /src/main/java/org/apache/hadoop/hbase/regionserver/ReadWriteConsistencyControl.java since this is a public constructor, add a comment that this initializes to 0 /src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java add javadoc to the public method and maybe turn it into a setter. i actually have a chance i'd like to get in eventually that requires disabling rwcc for a specified read /src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java this javadoc is stale now /src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java just remove - Jonathan On 2011-10-05 19:18:51, Amitanand Aiyer wrote: bq. bq. ----------------------------------------------------------- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2224/ bq. ----------------------------------------------------------- bq. bq. (Updated 2011-10-05 19:18:51) bq. bq. bq. Review request for Ted Yu, Michael Stack, Kannan Muthukkaruppan, and Karthik Ranganathan. bq. bq. bq. Summary bq. ------- bq. bq. address the 2856 issues by writing the memstoreTS to the disk. bq. bq. version v11 of the patch. bq. bq. uploading it here for easier review process. bq. bq. bq. This addresses bug HBASE-2856. bq. https://issues.apache.org/jira/browse/HBASE-2856 bq. bq. bq. Diffs bq. ----- bq. bq. /src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java 1174515 bq. /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 1174515 bq. /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 1174515 bq. /src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 1175027 bq. /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1174515 bq. /src/main/java/org/apache/hadoop/hbase/regionserver/ReadWriteConsistencyControl.java 1174515 bq. /src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java 1174515 bq. /src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1174515 bq. /src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 1174515 bq. /src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java 1174515 bq. /src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java 1175027 bq. /src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java 1174515 bq. /src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java 1174515 bq. /src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 1174515 bq. bq. Diff: https://reviews.apache.org/r/2224/diff bq. bq. bq. Testing bq. ------- bq. bq. mvn test bq. bq. bq. Thanks, bq. bq. Amitanand bq. bq. > TestAcidGuarantee broken on trunk > ---------------------------------- > > Key: HBASE-2856 > URL: https://issues.apache.org/jira/browse/HBASE-2856 > Project: HBase > Issue Type: Bug > Affects Versions: 0.89.20100621 > Reporter: ryan rawson > Assignee: Amitanand Aiyer > Priority: Blocker > Fix For: 0.94.0 > > Attachments: 2856-v2.txt, 2856-v3.txt, 2856-v4.txt, 2856-v5.txt, acid.txt > > > TestAcidGuarantee has a test whereby it attempts to read a number of columns from a row, and every so often the first column of N is different, when it should be the same. This is a bug deep inside the scanner whereby the first peek() of a row is done at time T then the rest of the read is done at T+1 after a flush, thus the memstoreTS data is lost, and previously 'uncommitted' data becomes committed and flushed to disk. > One possible solution is to introduce the memstoreTS (or similarly equivalent value) to the HFile thus allowing us to preserve read consistency past flushes. Another solution involves fixing the scanners so that peek() is not destructive (and thus might return different things at different times alas). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira