hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: NoServerForRegionException when intensive insertions
Date Sat, 11 Dec 2010 18:27:51 GMT
That region was offline a while probably because it was taking a while
for split daughters to come online.  If you do a Get in shell for the
start row, is it available now?  Grep in master for 1291950247501.
See if you can figure out a history of regions carrying this row.  See
if you see something like a region split that perhaps was taking a
while.  Go take a look at the regionserver log where split was
happening (The master in its logs should report the location of the
plit).  What was going on on this region?  Was it struggling?  GC
pause?  Swapping?

NotServingRegionException can be part of 'normal' operation.  The
server will throw the client this exception as signal that it should
recalibrate -- i.e. go back to .META. to find a regions new location
-- because the region has moved because of split or crash, etc.  if
the NSREs go on for too long, they turn from DEBUG into ERRORs and
client fails.  If loading on cluster is intensive, it can take a while
for regions to re-online. There could be another issue in the way of
the region re-onlining.  Grepping around in the logs as per above
should give a clue.

St.Ack

On Thu, Dec 9, 2010 at 10:00 PM, Tao Xie <xietao.mailbox@gmail.com> wrote:
> hi, all
>
> I met this exception when I doing intensive insertions using YCSB.  Anybody
> give me some clues on this? I use hbase 0.20.6.
>
> com.yahoo.ycsb.DBException:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
> region server  -- nothing found, no 'location' returned,
> tableName=usertable, reload=true -- for region , row 'user1001412274', but
> failed after 11 attempts.
> Exceptions:
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
>
>        at com.yahoo.ycsb.db.HBaseClient.cleanup(Unknown Source)
>        at com.yahoo.ycsb.DBWrapper.cleanup(Unknown Source)
>        at com.yahoo.ycsb.ClientThread.run(Unknown Source)
> Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying
> to contact region server  -- nothing found, no 'location' returned,
> tableName=usertable, reload=true -- for region , row 'user1001412274', but
> failed after 11 attempts.
> Exceptions:
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
> listed in .META. for region usertable,,1291950247501
>
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionLocationForRowWithRetries(HConnectionManager.java:1095)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.access$200(HConnectionManager.java:240)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers$Batch.getRegionName(HConnectionManager.java:1191)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers$Batch.process(HConnectionManager.java:1168)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:1238)
>        at
> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:666)
>        ... 3 more
>

Mime
View raw message