hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5060) HBase client is blocked forever
Date Mon, 19 Dec 2011 09:52:31 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172151#comment-13172151
] 

Hadoop QA commented on HBASE-5060:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12507874/HBASE-5060_trunk.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    -1 javadoc.  The javadoc tool appears to have generated -152 warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 76 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.mapred.TestTableMapReduce
                  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/540//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/540//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/540//console

This message is automatically generated.
                
> HBase client is blocked forever
> -------------------------------
>
>                 Key: HBASE-5060
>                 URL: https://issues.apache.org/jira/browse/HBASE-5060
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 0.90.4
>            Reporter: gaojinchao
>            Assignee: gaojinchao
>            Priority: Critical
>             Fix For: 0.92.0, 0.90.6
>
>         Attachments: HBASE-5060_Branch90trial.patch, HBASE-5060_trunk.patch
>
>
> Since the client had a temporary network failure, After it recovered.
> I found my client thread was blocked. 
> Looks below stack and logs, It said that we use a invalid CatalogTracker in function
"tableExists".
> Block stack:
> "WriteHbaseThread33" prio=10 tid=0x00007f76bc27a800 nid=0x2540 in Object.wait() [0x00007f76af4f3000]
>    java.lang.Thread.State: TIMED_WAITING (on object monitor)
>          at java.lang.Object.wait(Native Method)
>          at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:331)
>          - locked <0x00007f7a67817c98> (a java.util.concurrent.atomic.AtomicBoolean)
>          at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMetaServerConnectionDefault(CatalogTracker.java:366)
>          at org.apache.hadoop.hbase.catalog.MetaReader.tableExists(MetaReader.java:427)
>          at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:164)
>          at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source)
>          at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source)
>          - locked <0x00007f7a4c5dc578> (a com.huawei.hdi.hbase.HbaseReOper)
>          at com.huawei.hdi.hbase.HbaseFileOperate.writeToHbase(Unknown Source)
>          at com.huawei.hdi.hbase.WriteHbaseThread.run(Unknown Source)
> In ZooKeeperNodeTracker, We don't throw the KeeperException to high level.
> So in CatalogTracker level, We think ZooKeeperNodeTracker start success and
> continue to process .
> [WriteHbaseThread33]2011-12-16 17:07:33,153[WARN ]  | hconnection-0x334129cf6890051-0x334129cf6890051-0x334129cf6890051
Unable to get data of znode /hbase/root-region-server | org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:557)
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
for /hbase/root-region-server
>          at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
>          at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>          at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:931)
>          at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:549)
>          at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:73)
>          at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:136)
>          at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:111)
>          at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:162)
>          at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source)
>          at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source)
>          at com.huawei.hdi.hbase.HbaseFileOperate.writeToHbase(Unknown Source)
>          at com.huawei.hdi.hbase.WriteHbaseThread.run(Unknown Source)
> [WriteHbaseThread33]2011-12-16 17:07:33,361[ERROR]  | hconnection-0x334129cf6890051-0x334129cf6890051-0x334129cf6890051
Received unexpected KeeperException, re-throwing exception | org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.keeperException(ZooKeeperWatcher.java:385)
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
for /hbase/root-region-server
>          at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
>          at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>          at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:931)
>          at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:549)
>          at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:73)
>          at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:136)
>          at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:111)
>          at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:162)
>          at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source)
>          at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source)
>          at com.huawei.hdi.hbase.HbaseFileOperate.writeToHbase(Unknown Source)
>          at com.huawei.hdi.hbase.WriteHbaseThread.run(Unknown Source)
> [WriteHbaseThread33]2011-12-16 17:07:33,361[FATAL]  | Unexpected exception during initialization,
aborting | org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.abort(HConnectionManager.java:1351)
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
for /hbase/root-region-server
>          at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
>          at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>          at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:931)
>          at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:549)
>          at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:73)
>          at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:136)
>          at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:111)
>          at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:162)
>          at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source)
>          at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source)
>          at com.huawei.hdi.hbase.HbaseFileOperate.writeToHbase(Unknown Source)
>          at com.huawei.hdi.hbase.WriteHbaseThread.run(Unknown Source)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message