hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1910) IndexedRegion RPC deadlock
Date Fri, 16 Oct 2009 01:40:31 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Purtell updated HBASE-1910:
----------------------------------

    Attachment: thread.dump.gz

Attached thread dump provided by user for reference.

> IndexedRegion RPC deadlock
> --------------------------
>
>                 Key: HBASE-1910
>                 URL: https://issues.apache.org/jira/browse/HBASE-1910
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.1, 0.21.0
>            Reporter: Andrew Purtell
>             Fix For: 0.22.0
>
>         Attachments: thread.dump.gz
>
>
> From Tatsuya Kawano up on hbase-user@
> {quote}
> 50 client threads who try to put millions of records, autoFlush(false), flushCommits()
on every 5,000 put. After inserting about 3 million records, a deadlock occurred on a region
server who has both the table and index regions loaded.
> I have attached a full thread dump of the deadlocked region server, and you can see IPC
Server handlers are blocked in org.apache.hadoop.hbase.regionserver.tableindexed.IndexedRegion.updateIndex().
> I found the flowing FIXME comment on updateIndex() method, and it seems this is the deadlock
I'm having.
> {code}
>   // FIXME: This call takes place in an RPC, and requires an RPC. This makes for
>   // a likely deadlock if the number of RPCs we are trying to serve is >= the
>   // number of handler threads.
>   private void updateIndex(IndexSpecification indexSpec, byte[] row,
>       SortedMap<byte[], byte[]> columnValues) throws IOException {
> {code}
> I use HBase 0.20.1 and my region servers were running with 10 RPC handler threads on
each (default).
> Maybe you can workaround this by adding more RPC handlers (increase the value of "hbase.regionserver.handler.count"
in hbase-site.xml)
> {quote}
> Opening this issue to track the FIXME.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message