hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1409) table TTL expires before its real expiration time which cause row key disappear from Scanner
Date Wed, 20 May 2009 21:52:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711375#action_12711375

Andrew Purtell commented on HBASE-1409:

Because the problem happens < 24 hours, major compaction is not a factor. Checked code
paths for expiring values prior to rewrite (and also checked compaction code). No apparent
defects. Interesting that scanner is reported to be affected but get or getRow is reported
to not. Gave StoreFileScanner an especially close look.

One possibility is that initial timestamp is erroneous, leading to proper expiration against
user expectation. I checked the cluster this problem was reported on and all clocks are synchronized,
but cannot say if this also held at the time the problem was reported.

Some simple test case does not reproduce.

> table TTL expires before its real expiration time which cause row key disappear from
> --------------------------------------------------------------------------------------------
>                 Key: HBASE-1409
>                 URL: https://issues.apache.org/jira/browse/HBASE-1409
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.19.1
>         Environment: Linux jdc2-atr-dc-2 2.6.18-92.1.22.el5 #1 SMP Tue Dec 16 11:57:43
EST 2008 x86_64 x86_64 x86_64 GNU/Linux
> Hadoop 0.19.1 (r745977) and HBase 0.19.1 (r745977) release candidate.
>            Reporter: Andy Lee
>            Assignee: Andrew Purtell
>   Original Estimate: 120h
>  Remaining Estimate: 120h
> When we create a table with the following schema:
> {NAME => 'jobs_global', IS_ROOT => 'false', IS_META => 'false', MAX_FILESIZE
=> '134217728', FAMILIES => [{NAM
> E => 'job', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '1',
LENGTH => '2147483647', TTL => '86
> 400', IN_MEMORY => 'false', BLOCKCACHE => 'false'}], INDEXES => []}
> The TTL is set to 86400 which should expire after 1 day, but the truth is that it expired
before 86400 seconds.
> To reproduce, create a table with the above schema and run some stress testing to create
some splits and compaction,
> usually in 4 - 5 hours, the row key will start missing from the Scanners.
> By invoking HTable.get() and HTable.getRow(), the column appears to exist.
> But if you launch a scanner or a MapReduce task to scan the table, the key will be missing.
> By running a simple MapReduce task that prints out all the key value, you can tell some
keys are already missing prior to its expiration time.
> When we alter the table's TTL to a longer time, e.g. 604800, the row key appears in the

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message