hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1058) Prevent runaway compactions
Date Tue, 14 Apr 2009 07:33:15 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Purtell updated HBASE-1058:

    Attachment: hbase-1058-v4.patch

v4 patch goes back to holding up the flush rather than failing it (which has bad effects).
Does not try to hold up flush indefinitely, either. 


Yes the intent of the patch is to delay flushing if there are too many store files while simultaneously
requesting a compaction on the region. Otherwise a backlog of storefiles may build up under
very heavy write load such that any HRS that tries to compact will OOME. It does indeed mean
the client is more likely to be blocked. 

Right now we tie the determination of what are too many store files to the value of hbase.hstore.compactionThreshold
but it can be a separate tunable. 

I only observe this triggered during very heavy write load.

> Prevent runaway compactions
> ---------------------------
>                 Key: HBASE-1058
>                 URL: https://issues.apache.org/jira/browse/HBASE-1058
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: Andrew Purtell
>            Priority: Blocker
>             Fix For: 0.20.0
>         Attachments: hbase-1058-v2.patch, hbase-1058-v4.patch, hbase-1058.patch
> A rabid upload will easily outrun our compaction ability dropping flushes faster than
we can compact them up.  Fix.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message