hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Qiang Tian (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11368) Multi-column family BulkLoad fails if compactions go on too long
Date Wed, 08 Oct 2014 03:43:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163024#comment-14163024
] 

Qiang Tian commented on HBASE-11368:
------------------------------------

As [~stack] mentioned in http://search-hadoop.com/m/DHED4NR0wT, the HRegion#lock is to protect
region close. the comments in HRegion.java and the fact that only HRegion#doClose locks the
writelock(if we do not consider HRegion#startBulkRegionOperation) also show that.

so using HRegion#lock to protect multi-CF bulkload in HBASE-4552 looks too heavy-weight?
from the stacktrace of HBASE-10882, all the read/scan are blocked since bulkload is waiting
for lock.writelock, however compaction already acquired lock.readlock and is reading data,
a time-consuming operation.

and related topic is discussed again in http://search-hadoop.com/m/DHED4I11p31. perhaps we
need another region level lock.











> Multi-column family BulkLoad fails if compactions go on too long
> ----------------------------------------------------------------
>
>                 Key: HBASE-11368
>                 URL: https://issues.apache.org/jira/browse/HBASE-11368
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>
> Compactions take a read lock.  If a multi-column family region, before bulk loading,
we want to take a write lock on the region.  If the compaction takes too long, the bulk load
fails.
> Various recipes include:
> + Making smaller regions (lame)
> + [~victorunique] suggests major compacting just before bulk loading over in HBASE-10882
as a work around.
> Does the compaction need a read lock for that long?  Does the bulk load need a full write
lock when multiple column families?  Can we fail more gracefully at least?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message