hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11368) Multi-column family BulkLoad fails if compactions go on too long
Date Fri, 04 Sep 2015 00:36:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730105#comment-14730105
] 

Nick Dimiduk commented on HBASE-11368:
--------------------------------------

Looks like HBASE-6028 wants to implement the meat of what I've proposed above. It also happens
to be 2/3 of the work for HBASE-12446. Seems like good bang for the buck on this approach.

Chatting with [~enis] and [~devaraj] about this offline. Another idea is we can reduce the
scope of when the read lock is held during compaction. In theory the compactor only needs
a region read lock while deciding what files to compact and at the time of committing the
compaction. We're protected from the case of region close events because compactions are checking
(between every Cell!) if the store has been closed in order to abort in such a case. Is there
another reason why we would want to hold the read lock for the entire duration of the compaction?
[~stack] [~lhofhansl]?

> Multi-column family BulkLoad fails if compactions go on too long
> ----------------------------------------------------------------
>
>                 Key: HBASE-11368
>                 URL: https://issues.apache.org/jira/browse/HBASE-11368
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: Qiang Tian
>         Attachments: hbase-11368-0.98.5.patch, hbase11368-master.patch, key_stacktrace_hbase10882.TXT,
performance_improvement_verification_98.5.patch
>
>
> Compactions take a read lock.  If a multi-column family region, before bulk loading,
we want to take a write lock on the region.  If the compaction takes too long, the bulk load
fails.
> Various recipes include:
> + Making smaller regions (lame)
> + [~victorunique] suggests major compacting just before bulk loading over in HBASE-10882
as a work around.
> Does the compaction need a read lock for that long?  Does the bulk load need a full write
lock when multiple column families?  Can we fail more gracefully at least?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message