hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Assigned: (HBASE-2990) It's possible for regions to not be major compacted for more than hbase.hregion.majorcompaction
Date Sat, 30 Oct 2010 20:01:20 GMT

     [ https://issues.apache.org/jira/browse/HBASE-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jonathan Gray reassigned HBASE-2990:

    Assignee: Jonathan Gray

I have a patch up on rb for this.  Also worked on a second style implementation that uses
metadata in HFiles but I care for it even less than the HRegionData approach.

I'm going to give another go w/ HRegionData and utilize versioning in HRI so we're properly
backwards compatible.  Will have a patch up in the next couple days.

> It's possible for regions to not be major compacted for more than hbase.hregion.majorcompaction
> -----------------------------------------------------------------------------------------------
>                 Key: HBASE-2990
>                 URL: https://issues.apache.org/jira/browse/HBASE-2990
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jonathan Gray
>            Priority: Blocker
>             Fix For: 0.90.0
> This is something I've seen here since we upgraded to 0.89 since Gets are now Scans,
although I don't currently have any strong evidence that it wasn't happening in 0.20.
> I saw this when we began getting alerts on the frontend that some requests were taking
more than 8 seconds to complete. Even getting a value could take more than 3 minutes in the
shell. The first thing I did was major compacting the table that was slow and the problem
went away immediately. Looking in the logs, it seems the compaction transformed 2 files of
(total) 550MB into 5.2MB. Looking back in the logs for September, it appears that that table
was never major compacted and was slowly growing everyday. Some more grepping around showed
that quite a few regions were never major compacted.
> I'm still looking at the code, but the issue seems to be that the minor compactions are
always happening on all store files more than once per day on certain regions, meaning that
the oldest timestamp is always smaller than hbase.hregion.majorcompaction and major compactions
are never triggered.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message