Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <2126194800.1242856365717.JavaMail.jira@brutus>
Date: Wed, 20 May 2009 14:52:45 -0700 (PDT)
From: "Andrew Purtell (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-1409) table TTL expires before its real
 expiration time which cause row key disappear from Scanner
In-Reply-To: <1783431988.1242141585686.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711375#action_12711375 ] 

Andrew Purtell commented on HBASE-1409:
---------------------------------------

Because the problem happens < 24 hours, major compaction is not a factor. Checked code paths for expiring values prior to rewrite (and also checked compaction code). No apparent defects. Interesting that scanner is reported to be affected but get or getRow is reported to not. Gave StoreFileScanner an especially close look.

One possibility is that initial timestamp is erroneous, leading to proper expiration against user expectation. I checked the cluster this problem was reported on and all clocks are synchronized, but cannot say if this also held at the time the problem was reported.

Some simple test case does not reproduce.


> table TTL expires before its real expiration time which cause row key disappear from Scanner
> --------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1409
>                 URL: https://issues.apache.org/jira/browse/HBASE-1409
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.19.1
>         Environment: Linux jdc2-atr-dc-2 2.6.18-92.1.22.el5 #1 SMP Tue Dec 16 11:57:43 EST 2008 x86_64 x86_64 x86_64 GNU/Linux
> Hadoop 0.19.1 (r745977) and HBase 0.19.1 (r745977) release candidate.
>            Reporter: Andy Lee
>            Assignee: Andrew Purtell
>   Original Estimate: 120h
>  Remaining Estimate: 120h
>
> When we create a table with the following schema:
> {NAME => 'jobs_global', IS_ROOT => 'false', IS_META => 'false', MAX_FILESIZE => '134217728', FAMILIES => [{NAM
> E => 'job', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '1', LENGTH => '2147483647', TTL => '86
> 400', IN_MEMORY => 'false', BLOCKCACHE => 'false'}], INDEXES => []}
> The TTL is set to 86400 which should expire after 1 day, but the truth is that it expired before 86400 seconds.
> To reproduce, create a table with the above schema and run some stress testing to create some splits and compaction,
> usually in 4 - 5 hours, the row key will start missing from the Scanners.
> By invoking HTable.get() and HTable.getRow(), the column appears to exist.
> But if you launch a scanner or a MapReduce task to scan the table, the key will be missing.
> By running a simple MapReduce task that prints out all the key value, you can tell some keys are already missing prior to its expiration time.
> When we alter the table's TTL to a longer time, e.g. 604800, the row key appears in the scanner.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.