hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: Help needed: failed unit test based on miniMRCluster on Yarn with cdh 5.1.3
Date Fri, 13 Feb 2015 10:43:53 GMT
If you can replicate this on the ASF 2.6.0 release then it's something to raise a JIRA on @
issues.apache.org



On 13 February 2015 at 00:47:18, Lumin Zhang (lumzhang@adobe.com<mailto:lumzhang@adobe.com>)
wrote:

Hi, folks,

I have encountered a strange error when I am trying to upgrade from cdh4 to cdh5.1.3.

I have a unit test based on miniMRcluster and ClusterMapReduceTestCase. It runs successfully
with cdh 4. After I have changed my maven setting to upgrade to cdh5. The running always failed.

I have tried to isolate the problem. So I used the standard unit test (https://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java)
and compile it with my maven setting, then I reproduced the same error by only testing testRestart().


22596 [main] INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory - Stopping JobHistory

22596 [main] INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory - Stopping History Cleaner/Move
To Done

22597 [Thread[Thread-184,5,main]] ERROR org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager
- ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted

22597 [main] INFO org.apache.hadoop.hdfs.MiniDFSCluster - Shutting down the Mini HDFS Cluster

22597 [main] INFO org.apache.hadoop.hdfs.MiniDFSCluster - Shutting down DataNode 1

22598 [main] WARN org.apache.hadoop.hdfs.server.datanode.DirectoryScanner - DirectoryScanner:
shutdown has been called


The test should be compatible for both MR1 and MR2. I also tried to overwrite some Yarn configuration,
but it didn't help. I tried to google the related issue, but no luck for this particular issue.Any
comments or Help are appreciated. Thanks so much!


My mvn configuration part is as follows,


<version.hadoop>2.3.0-cdh5.1.3</version.hadoop>

<version.hadoop.core>2.3.0-cdh5.1.3</version.hadoop.core>

<version.hadoop.minicluster>2.3.0-cdh5.1.3</version.hadoop.minicluster>


<dependency>

<groupId>org.apache.hadoop</groupId>

<artifactId>hadoop-client</artifactId>

<version>${version.hadoop}</version>

</dependency>


<dependency>

<groupId>org.apache.hadoop</groupId>

<artifactId>hadoop-minicluster</artifactId>

<version>${version.hadoop.minicluster}</version>

<exclusions>

<exclusion>

<groupId>org.slf4j</groupId>

<artifactId>slf4j-log4j12</artifactId>

</exclusion>

</exclusions>

</dependency>


<dependency>

<groupId>${project.groupId}</groupId>

<artifactId>common-hadoop</artifactId>

<version>${project.version}</version>

</dependency>




Best regards,

Lumin Zhang


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message