hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3214) TestMasterFailover.testMasterFailoverWithMockedRITOnDeadRS is failing
Date Wed, 10 Nov 2010 02:12:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930441#action_12930441
] 

Jonathan Gray commented on HBASE-3214:
--------------------------------------

This is what I see locally as the failure:

{noformat}
2010-11-09 18:07:17,291 FATAL [Master:0;172.24.154.154:56721] master.HMaster(888): Unhandled
exception. Starting shutdown.
java.lang.RuntimeException: Failed exists test on hdfs://localhost:56643/user/jgray/.logs
	at org.apache.hadoop.hbase.master.MasterFileSystem.splitLogAfterStartup(MasterFileSystem.java:162)
	at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:374)
	at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:272)
	at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.IOException: Filesystem closed
	at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:232)
	at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:623)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:453)
	at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:648)
	at org.apache.hadoop.hbase.master.MasterFileSystem.splitLogAfterStartup(MasterFileSystem.java:158)
	... 3 more
{noformat}

Somehow DFS is being closed.  No idea why this is happening now but wasn't before.

Shortly before this exception, I see this log line:
{noformat}
2010-11-09 18:07:12,413 INFO  [Shutdown of DFS[DFSClient[clientName=DFSClient_hb_rs_172.24.154.154,56663,1289354815892_1289354817252,
ugi=jgray.hfs.1,supergroup]]] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(248):
Hook closing fs=DFS[DFSClient[clientName=DFSClient_hb_rs_172.24.154.154,56663,1289354815892_1289354817252,
ugi=jgray.hfs.1,supergroup]]
{noformat}

Looks like an RS exiting is now triggering a complete shutdown of DFS.

If I comment out the below line in MiniHBaseCluster line 189, the test passes.
{noformat}
      this.shutdownThread = new SingleFileSystemShutdownThread(getFileSystem());
{noformat}

What has changed?  In unit tests, if RS is being shut down, should not take the entire FS
with it?

> TestMasterFailover.testMasterFailoverWithMockedRITOnDeadRS is failing
> ---------------------------------------------------------------------
>
>                 Key: HBASE-3214
>                 URL: https://issues.apache.org/jira/browse/HBASE-3214
>             Project: HBase
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.90.0
>            Reporter: Jonathan Gray
>             Fix For: 0.90.0
>
>
> Failing on hudson and locally

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message