hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5328) MiniDFSCluster times out at startup
Date Wed, 09 Oct 2013 08:26:42 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13790162#comment-13790162
] 

Steve Loughran commented on HDFS-5328:
--------------------------------------

not seen this -is the jenkins box under load/memory swap?

regardless, hard coded delays are always brittle, and could be made configurable with a longer
timeout. After all, if the minicluster comes up normally the test is no slower, and if it
takes >10s to start, the test fails which is far more expensive in terms of human intervention.

Maybe extend the default spinup time to 30s?

> MiniDFSCluster times out at startup
> -----------------------------------
>
>                 Key: HDFS-5328
>                 URL: https://issues.apache.org/jira/browse/HDFS-5328
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 2.1.1-beta
>            Reporter: Alejandro Abdelnur
>
> It seems MiniDFSCluster startup got slower lately and we are observing the following
intermittent failure on startup in our jenkins boxes:
> {code}
> java.io.IOException: Timed out waiting for Mini HDFS Cluster to start
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.waitClusterUp(MiniDFSCluster.java:972)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:665)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:585)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:511)
> {code}
> By looking at the code the {{waitClusterUp()}} has a hardcoded 10secs timeout (10 1 sec
waits).
> There are two things we should look at it, make the timeout configurable, and more important,
see what has changed to make the startup slower.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message