hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars George (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-2021) Add compaction details to master UI
Date Tue, 01 Dec 2009 18:20:21 GMT
Add compaction details to master UI

                 Key: HBASE-2021
                 URL: https://issues.apache.org/jira/browse/HBASE-2021
             Project: Hadoop HBase
          Issue Type: Improvement
            Reporter: Lars George
            Assignee: Lars George
            Priority: Minor

There are two issues with this, first to detect that there is a compaction needed. You can
currently use the little helper util that checks if a table has at least one colfam with more
than one store file. I though about scanning all tables and all colfams in each and then compute
the "fragmentation" ratio as a percentage of colfams with more than one store to the total
number of colfams. That gives a "Table xyz is 33% fragmented" output. While minor percentage
are normal under insert operations it is still important to know how bad the fragmentation
is overall.

Another idea is to weigh the number of files per store too, so that if you have two per colfam
it is considered "low" and if you have more, for example 6-8 it is considered "high". Not
sure how that can be done yet but noting the idea down here.

Of course seeing the .META. fragmentation is useful to quickly debug performance issues (as
JD told me on IRC).

The other issue is that when you have started a compaction you have no idea how far it is
and if it is still in progress. One indication of course is the above value. If it is 0% then
all is done. But if you are at say 23%, is it still compacting? We could have a simple status
that compactions are still in progress.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message