hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Appy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15959) Fix flaky test TestRegionServerMetrics.testMobMetrics
Date Wed, 08 Jun 2016 03:32:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319926#comment-15319926

Appy commented on HBASE-15959:

bq. region.compact doesn't compact mob files, I think this is not related with cleaners.

If it's moving cells from mob to store, doesn't that mean it's compacting mob files too. [Here|https://github.com/apache/hbase/blob/7efb9edecbdf8b35046230575d504e4caeb80f34/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java#L213],
we are moving mob cells to store or new writer.

You're right, first two parts are very clear what they are doing. Set the thresholds, expect
things to move around.

bq. In the 2nd compaction, the threshold is changed to 0, and all cells are moved to mob files
from store files. And in the second scan, the cells are the reference ones.

Cells 0-4 got moved to store file in the first compaction. And that's why in the [second scan|https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java#L441],
there is [no mob cell|https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java#L445].
 And we write cells 5-9 before [threshold is ste to 0|https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java#L453],
so they should be non-mob cells too.
So when the second compaction kicks, I would expect them to be non-reference cells. It's only
after the compaction has finished, that they'll become the reference cells again.
(Just to be sure, reference cells = mob cells, right?)

> Fix flaky test TestRegionServerMetrics.testMobMetrics
> -----------------------------------------------------
>                 Key: HBASE-15959
>                 URL: https://issues.apache.org/jira/browse/HBASE-15959
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Appy
>            Assignee: huaxiang sun
> It flakes [here|https://github.com/apache/hbase/blob/b557f0bec62a48753e5d01d7a47f3c9e5a6b3ee8/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java#L460].
> There are two weird things i identified:
> 1. In second compaction, [scanner|https://github.com/apache/hbase/blob/b557f0bec62a48753e5d01d7a47f3c9e5a6b3ee8/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java#L173]
has 10 storefiles. Shouldn't there be 6? 5 from recent flushes and 1 from earlier compaction.
Probably because mob cleaner doesn't clean old hfiles. Does this needs fixing?
> 2. Across runs, same cell (ie. same key) may or may not be considered mob reference cell.
This at least happens with row keys 0 - 4 (which got compacted earlier). [~jmhsieh] Any ideas
why this would happen.

This message was sent by Atlassian JIRA

View raw message