hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS fails on Jenkins
Date Fri, 04 Nov 2011 03:55:43 GMT
Hi,
Currently TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS <
https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/105/testReport/org.apache.hadoop.hbase.master/TestMasterFailover/testMasterFailoverWithMockedRITOnDeadRS/<https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/lastCompletedBuild/testReport/org.apache.hadoop.hbase.master/TestMasterFailover/testMasterFailoverWithMockedRITOnDeadRS/>>
consistently fails on 0.92 and TRUNK.

I intended to log a JIRA but https://issues.apache.org is giving me 503
error.

I briefly went over the code.
I think after each region is added to regionsThatShouldBeOnline, we should
log the name of region:
    // Region of enabled on dead server gets closed but not ack'd by master
    region = enabledAndOnDeadRegions.remove(0);
    regionsThatShouldBeOnline.add(region);
    log("2. expecting " + region.toString() + " to be online: ");

so that if the assertion below fails we know what type of scenario wasn't
working:
    for (HRegionInfo hri : regionsThatShouldBeOnline) {
      assertTrue("region=" + hri.getRegionNameAsString(),
onlineRegions.contains(hri));
    }

>From the above mentioned test output I saw a lot of:

2011-11-03 21:52:58,652 FATAL [Thread-558.logSyncer] wal.HLog(1106):
Could not sync. Requesting close of hlog
java.io.IOException: Reflection
	at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:225)
	at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1090)
	at org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1194)
	at org.apache.hadoop.hbase.regionserver.wal.HLog$LogSyncer.run(HLog.java:1056)
	at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:223)
	... 4 more
Caused by: java.io.IOException: DFSOutputStream is closed
	at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.sync(DFSClient.java:3483)
	at org.apache.hadoop.fs.FSDataOutputStream.sync(FSDataOutputStream.java:97)
	at org.apache.hadoop.io.SequenceFile$Writer.syncFs(SequenceFile.java:944)
	... 8 more

Maybe they have something to do with regions stuck in RIT.

Cheers

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message