hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "huaxiang sun (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15959) Fix flaky test TestRegionServerMetrics.testMobMetrics
Date Wed, 08 Jun 2016 02:06:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319858#comment-15319858
] 

huaxiang sun commented on HBASE-15959:
--------------------------------------

I have a patch ready. The root cause is
{code}
      assertCounter("cellsCountCompactedToMob", 2 * numHfiles);
{code}

Assumes that there is no mob reference cells. Actually, after the first major compact, the
compacted files are still there in the family directory (compaction.discharger does not get
chance to kick in yet). When scan for the next major compaction, these files may still be
used, so scan may hit the cells in these compacted files (hence it has mob reference). 

The fix is to wait compaction.discharger to move compacted files to archive and then start
the second major compaction. 

I will upload a patch after some clean up.


> Fix flaky test TestRegionServerMetrics.testMobMetrics
> -----------------------------------------------------
>
>                 Key: HBASE-15959
>                 URL: https://issues.apache.org/jira/browse/HBASE-15959
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Appy
>            Assignee: huaxiang sun
>
> It flakes [here|https://github.com/apache/hbase/blob/b557f0bec62a48753e5d01d7a47f3c9e5a6b3ee8/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java#L460].
> There are two weird things i identified:
> 1. In second compaction, [scanner|https://github.com/apache/hbase/blob/b557f0bec62a48753e5d01d7a47f3c9e5a6b3ee8/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java#L173]
has 10 storefiles. Shouldn't there be 6? 5 from recent flushes and 1 from earlier compaction.
Probably because mob cleaner doesn't clean old hfiles. Does this needs fixing?
> 2. Across runs, same cell (ie. same key) may or may not be considered mob reference cell.
[here|https://github.com/apache/hbase/blob/b557f0bec62a48753e5d01d7a47f3c9e5a6b3ee8/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java#L213].
This at least happens with row keys 0 - 4 (which got compacted earlier). [~jmhsieh] Any ideas
why this would happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message