hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Latham (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-2770) Major compactions from shell may not major compact all families
Date Tue, 22 Jun 2010 22:04:51 GMT
Major compactions from shell may not major compact all families
---------------------------------------------------------------

                 Key: HBASE-2770
                 URL: https://issues.apache.org/jira/browse/HBASE-2770
             Project: HBase
          Issue Type: Bug
    Affects Versions: 0.20.4
            Reporter: Dave Latham
             Fix For: 0.20.6


As part of a data center migration, I initiated a major_compaction request on all tables from
the shell.  A few hours later, all the region servers in the cluster appeared to have completed
the compactions and all compactionQueue metrics were back to 0.  However, some column families
of some regions had not actually done a major compaction.

Digging through logs and code, it looks like the following happened.  The shell makes a major
compaction request which sets HRegion.forceMajorCompaction to true for every region.  Periodically,
the HRegionServer.MajorCompactionChecker checks to see if a major compaction is needed in
any family's store.  If so, calls CompactSplitThread.compactionRequested which ends up setting
the region forceMajorCompaction to false, even if it is already in the compaction queue and
set to true.  Then, when that region comes off the queue to be compacted, each family/store
separately checks for whether it should do a major compaction, so some families may not do
so.

(This is not good if, for example, you're doing a DistCp of the hbase dir and later on the
cluster decides to do a compaction on those files and deletes ones the DistCp job is looking
for, causing it to fail.)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message