hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8060) "Num compacting KVs" diverges from "num compacted KVs" over time
Date Wed, 19 Jun 2013 01:40:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687502#comment-13687502
] 

Sergey Shelukhin commented on HBASE-8060:
-----------------------------------------

Can you elaborate on the test used? There is a possible culprit from skimming the code.
Due to optimizations in ScanQueryMatcher compaction may not actually observe all KVs if there
are updates and deletes present, some KVs/rows/columns may be skipped. So kvs compacted will
never reach the total. Compaction progress is only reset when compaction is started so this
"unfinished" progress will stick around for some time.
However in standard LoadTestTool there are no updates and this should not happen.
Did the test have updates or deletes?

I will try to repro later this week, to see what is going on.
                
> "Num compacting KVs" diverges from "num compacted KVs" over time
> ----------------------------------------------------------------
>
>                 Key: HBASE-8060
>                 URL: https://issues.apache.org/jira/browse/HBASE-8060
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction, UI
>    Affects Versions: 0.94.6, 0.95.0, 0.95.2
>            Reporter: Andrew Purtell
>            Assignee: Sergey Shelukhin
>         Attachments: screenshot.png
>
>
> I have been running what amounts to an ingestion test for a day or so. This is an all-in-one
cluster launched with './bin/hbase master start' from sources. In the RS stats on the master
UI, the "num compacting KVs" has diverged from "num compacted KVs" even though compaction
has been completed from perspective of selection, no compaction tasks are running on the RS.
I think this could be confusing -- is compaction happening or not?
> Or maybe I'm misunderstanding what this is supposed to show?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message