hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10767) Load balancer may interfere with tests in TestHBaseFsck
Date Mon, 17 Mar 2014 19:51:43 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13938288#comment-13938288
] 

Hudson commented on HBASE-10767:
--------------------------------

FAILURE: Integrated in HBase-TRUNK #5020 (See [https://builds.apache.org/job/HBase-TRUNK/5020/])
HBASE-10767 Load balancer may interfere with tests in TestHBaseFsck (tedyu: rev 1578447)
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


> Load balancer may interfere with tests in TestHBaseFsck
> -------------------------------------------------------
>
>                 Key: HBASE-10767
>                 URL: https://issues.apache.org/jira/browse/HBASE-10767
>             Project: HBase
>          Issue Type: Test
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>             Fix For: 0.99.0, 0.98.2
>
>         Attachments: 10767-v1.txt
>
>
> When analyzing test failure shown in https://builds.apache.org/job/HBase-TRUNK/5018/testReport/org.apache.hadoop.hbase.util/TestHBaseFsck/testHbckThreadpooling/
, I saw the following events:
> {code}
> 2014-03-16 04:02:05,721 INFO  [juno.apache.org,41286,1394942220308-BalancerChore] master.HMaster(1490):
balance hri=NoHdfsTable,,1394942287079.                             f66b7b32c7bfed7cc8637ee9b033ef14.,
src=juno.apache.org,37923,1394942221817, dest=juno.apache.org,57897,1394942221876
> 2014-03-16 04:02:05,721 DEBUG [juno.apache.org,41286,1394942220308-BalancerChore] master.AssignmentManager(2239):
Starting unassign of NoHdfsTable,,1394942287079.          f66b7b32c7bfed7cc8637ee9b033ef14.
(offlining), current state: {f66b7b32c7bfed7cc8637ee9b033ef14 state=OPEN, ts=1394942287774,
server=juno.apache.org,37923,1394942221817}
> ...
> 2014-03-16 04:02:05,742 DEBUG [juno.apache.org,41286,1394942220308-BalancerChore] master.AssignmentManager(1704):
Offline NoHdfsTable,,1394942287079.f66b7b32c7bfed7cc8637ee9b033ef14., it's not any more on
juno.apache.org,37923,1394942221817
> org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException:
The region f66b7b32c7bfed7cc8637ee9b033ef14 is not online, and is not opening.
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:2617)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:3796)
> ...
> 2014-03-16 04:02:05,754 DEBUG [juno.apache.org,41286,1394942220308-BalancerChore] master.AssignmentManager(2191):
No previous transition plan found (or ignoring an existing plan) for NoHdfsTable,,1394942287079.f66b7b32c7bfed7cc8637ee9b033ef14.;
generated random plan=hri=NoHdfsTable,,1394942287079.f66b7b32c7bfed7cc8637ee9b033ef14., src=,
dest=juno.apache.org,57897,1394942221876; 3 (online=3, available=3) available servers, forceNewPlan=false
> ...
> 2014-03-16 04:02:05,786 DEBUG [RS_OPEN_REGION-juno:57897-0] regionserver.HRegion(4402):
Opening region: {ENCODED => f66b7b32c7bfed7cc8637ee9b033ef14, NAME => 'NoHdfsTable,,1394942287079.f66b7b32c7bfed7cc8637ee9b033ef14.',
STARTKEY => '', ENDKEY => 'A'}
> ...
> 2014-03-16 04:02:05,787 DEBUG [RS_OPEN_REGION-juno:57897-0] regionserver.HRegion(563):
Instantiated NoHdfsTable,,1394942287079.f66b7b32c7bfed7cc8637ee9b033ef14.
> ...
> 2014-03-16 04:02:06,862 DEBUG [pool-1-thread-1] util.HBaseFsck(1452): Loading region
dirs from hdfs://localhost:48141/user/jenkins/hbase/data/default/NoHdfsTable
> {code}
> Load balancer tried to balance region NoHdfsTable,,1394942287079.f66b7b32c7bfed7cc8637ee9b033ef14.
- possibly due to regionPlan generated when NoHdfsTable was still around.
> However juno.apache.org,37923,1394942221817 didn't serve this region any more. So balancer
moved this region to juno.apache.org,57897,1394942221876.
> At 04:02:05,787, region was instantiated on juno.apache.org,57897,1394942221876.
> Soon this region was picked up by loadHdfsRegionDirs called in HBaseFsck.
> This created an unexpected empty HbckInfo via the call to getOrCreateInfo.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message