hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesse Hallio (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-2191) Commit fails occasionally with NotServingRegionException in transactional hbase inserts
Date Sun, 07 Feb 2010 17:00:28 GMT
Commit fails occasionally with NotServingRegionException in transactional hbase inserts
---------------------------------------------------------------------------------------

                 Key: HBASE-2191
                 URL: https://issues.apache.org/jira/browse/HBASE-2191
             Project: Hadoop HBase
          Issue Type: Bug
          Components: contrib
    Affects Versions: 0.20.3
         Environment: OpenSolaris, java 1.6.0_10, pseudo-distributed setup; Hadoop 0.20.1
and HBase 0.20.3, single java client, Hadoop dfs.replication=1, HBase configured to use transactional
server.

            Reporter: Jesse Hallio


I'm testing a simple bulk upload to HBase. Single table, single family. The aim is to upload
200M+ lines into HBase. I've  tried to commit the upload in various batches, from 1 line at
time to 25000 lines at time. Each time, at seemingly random point, the upload fails in commit
with a org.apache.hadoop.hbase.NotServingRegionException

Failure stack trace from the client:
---
10/02/07 16:38:40 DEBUG transactional.TransactionManager: Begining transaction 5311384003652808027
10/02/07 16:38:40 DEBUG transactional.TransactionState: Adding new hregion [db,,1265553236583]
to transaction [5311384003652808027]
10/02/07 16:38:46 DEBUG transactional.TransactionManager: atempting to commit trasaction:
id: 5311384003652808027, particpants: 1
10/02/07 16:38:46 DEBUG transactional.TransactionManager: Region [db,,1265553236583] votes
to commit transaction 5311384003652808027
10/02/07 16:38:46 DEBUG transactional.TransactionManager: Commiting [5311384003652808027]
10/02/07 16:38:46 DEBUG transactional.TransactionManager: Committed transaction [5311384003652808027]
in [379]ms
10/02/07 16:38:46 DEBUG transactional.TransactionManager: Begining transaction -5274908504613584338
10/02/07 16:38:46 DEBUG transactional.TransactionState: Adding new hregion [db,,1265553236583]
to transaction [-5274908504613584338]
10/02/07 16:38:47 DEBUG client.HConnectionManager$TableServers: Removed db,,1265553236583
for tableName=db from cache because of \x00\x00\x01\x24\xAC\xC3\x98\x1C
10/02/07 16:38:47 DEBUG client.HConnectionManager$TableServers: Cached location for db,,1265553236583
is 1.1.1.1:43917
10/02/07 16:38:48 DEBUG client.HConnectionManager$TableServers: Removed db,,1265553236583
for tableName=db from cache because of \x00\x00\x01\x24\xAC\xC3\x98\x1C
10/02/07 16:38:48 DEBUG client.HConnectionManager$TableServers: Cached location for db,,1265553236583
is 1.1.1.1:43917
10/02/07 16:38:49 DEBUG client.HConnectionManager$TableServers: Removed db,,1265553236583
for tableName=db from cache because of \x00\x00\x01\x24\xAC\xC3\x98\x1C
10/02/07 16:38:49 DEBUG client.HConnectionManager$TableServers: locateRegionInMeta attempt
0 of 10 failed; retrying after sleep of 1000 because: No server address listed in .META. for
region db,\x00\x00\x01\x24\xAC\xA5\xEAB,1265553527794
10/02/07 16:38:49 DEBUG client.HConnectionManager$TableServers: Removed .META.,,1 for tableName=.META.
from cache because of db,\x00\x00\x01\x24\xAC\xC3\x98\x1C,99999999999999
10/02/07 16:38:49 DEBUG client.HConnectionManager$TableServers: Cached location for .META.,,1
is 1.1.1.1:43917
10/02/07 16:38:50 DEBUG client.HConnectionManager$TableServers: Cached location for db,\x00\x00\x01\x24\xAC\xA5\xEAB,1265553527794
is 1.1.1.1:43917
10/02/07 16:38:50 DEBUG transactional.TransactionState: Adding new hregion [db,\x00\x00\x01\x24\xAC\xA5\xEAB,1265553527794]
to transaction [-5274908504613584338]
10/02/07 16:38:51 DEBUG transactional.TransactionManager: atempting to commit trasaction:
id: -5274908504613584338, particpants: 2
10/02/07 16:38:51 DEBUG transactional.TransactionManager: Commit of transaction [-5274908504613584338]
was unsucsessful
org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.NotServingRegionException:
db,,1265553236583
        at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2266)
        at org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegionServer.getTransactionalRegion(TransactionalRegionServer.java:131)
        at org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegionServer.commitRequest(TransactionalRegionServer.java:179)
        at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)

        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:723)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:333)
        at $Proxy0.commitRequest(Unknown Source)
        at org.apache.hadoop.hbase.client.transactional.TransactionManager.prepareCommit(TransactionManager.java:94)
        at org.apache.hadoop.hbase.client.transactional.TransactionManager.tryCommit(TransactionManager.java:151)
        at dbloader.dbLoader.run2(dbLoader.java:181)
        at dbloader.dbLoader.main(dbLoader.java:24)
Exception in thread "main" org.apache.hadoop.hbase.client.transactional.CommitUnsuccessfulException:
org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.NotServingRegionException:
db,,1265553236583
        at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2266)
        at org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegionServer.getTransactionalRegion(TransactionalRegionServer.java:131)
        at org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegionServer.commitRequest(TransactionalRegionServer.java:179)
        at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)

        at org.apache.hadoop.hbase.client.transactional.TransactionManager.prepareCommit(TransactionManager.java:135)
        at org.apache.hadoop.hbase.client.transactional.TransactionManager.tryCommit(TransactionManager.java:151)
        at dbloader.dbLoader.run2(dbLoader.java:181)
        at dbloader.dbLoader.main(dbLoader.java:24)
Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.NotServingRegionException:
db,,1265553236583
        at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2266)
        at org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegionServer.getTransactionalRegion(TransactionalRegionServer.java:131)
        at org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegionServer.commitRequest(TransactionalRegionServer.java:179)
        at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)

        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:723)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:333)
        at $Proxy0.commitRequest(Unknown Source)
        at org.apache.hadoop.hbase.client.transactional.TransactionManager.prepareCommit(TransactionManager.java:94)
        ... 3 more


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message