hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10118) Major compact keeps deletes with future timestamps
Date Fri, 04 Apr 2014 00:52:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13959491#comment-13959491

Hudson commented on HBASE-10118:

FAILURE: Integrated in hbase-0.96 #376 (See [https://builds.apache.org/job/hbase-0.96/376/])
HBASE-10118 Major compact keeps deletes with future timestamps (Liu Shaohui) (sershe: rev
* /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
* /hbase/branches/0.96/src/main/docbkx/book.xml

> Major compact keeps deletes with future timestamps
> --------------------------------------------------
>                 Key: HBASE-10118
>                 URL: https://issues.apache.org/jira/browse/HBASE-10118
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction, Deletes, regionserver
>            Reporter: Max Lapan
>            Assignee: Liu Shaohui
>            Priority: Minor
>         Attachments: HBASE-10118-trunk-v1.diff, HBASE-10118-trunk-v2.diff, HBASE-10118-trunk-v3.diff
> Hello!
> During migration from HBase 0.90.6 to 0.94.6 we found changed behaviour in how major
compact handles delete markers with timestamps in the future. Before HBASE-4721 major compact
purged deletes regardless of their timestamp. Newer versions keep them in HFile until timestamp
not reached.
> I guess this happened due to new check in ScanQueryMatcher {{EnvironmentEdgeManager.currentTimeMillis()
- timestamp) <= timeToPurgeDeletes}}.
> This can be worked around by specifying large negative value in {{hbase.hstore.time.to.purge.deletes}}
option, but, unfortunately, negative values are pulled up to zero by Math.max in HStore.java.
> Maybe, we are trying to do something weird by specifing delete timestamp in future, but
HBASE-4721 definitely breaks old behaviour we rely on.
> Steps to reproduce this:
> {code}
> put 'test', 'delmeRow', 'delme:something', 'hello'
> flush 'test'
> delete 'test', 'delmeRow', 'delme:something', 1394161431061
> flush 'test'
> major_compact 'test'
> {code}
> Before major_compact we have two hfiles with the following:
> {code}
> first:
> K: delmeRow/delme:something/1384161431061/Put/vlen=5/ts=0
> second:
> K: delmeRow/delme:something/1394161431061/DeleteColumn/vlen=0/ts=0
> {code}
> After major compact we get the following:
> {code}
> K: delmeRow/delme:something/1394161431061/DeleteColumn/vlen=0/ts=0
> {code}
> In our installation, we resolved this by removing Math.max and setting hbase.hstore.time.to.purge.deletes
to Integer.MIN_VALUE, which purges delete markers, and it looks like a solution. But, maybe,
there are better approach.

This message was sent by Atlassian JIRA

View raw message