hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajith S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5858) TestDiskFailures.testLogDirsFailures fails on trunk
Date Sat, 12 Nov 2016 14:01:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15659702#comment-15659702
] 

Ajith S commented on YARN-5858:
-------------------------------

Thanks [~varun_saxena] for reporting this issue.
The LocalDirsHandlerService failed to detect the failure as its thread was interrupted from
the exception below
{code}Exception in thread "DiskHealthMonitor-Timer" org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
Could not initialize log dir /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/org.apache.hadoop.yarn.server.TestDiskFailures/org.apache.hadoop.yarn.server.TestDiskFailures-logDir-nm-0_1
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.initializeLogDir(ResourceLocalizationService.java:1391)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.initializeLogDirs(ResourceLocalizationService.java:1379)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.access$000(ResourceLocalizationService.java:147)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$2.onDirsChanged(ResourceLocalizationService.java:284)
	at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.checkDirs(DirectoryCollection.java:397)
	at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.checkDirs(LocalDirsHandlerService.java:470)
	at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.access$500(LocalDirsHandlerService.java:52)
	at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService$MonitoringTimerTask.run(LocalDirsHandlerService.java:166)
	at java.util.TimerThread.mainLoop(Timer.java:555)
	at java.util.TimerThread.run(Timer.java:505){code}
So, even the current run for disk failure detection failed and thus failing the test case
I suggest in {{org.apache.hadoop.yarn.server.TestDiskFailures.waitForDiskHealthCheck()}} we
can check if {{org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLastDisksCheckTime()}}
is updated, then only we can be sure if thread execution is successful and we can continue
with our assertions

> TestDiskFailures.testLogDirsFailures fails on trunk
> ---------------------------------------------------
>
>                 Key: YARN-5858
>                 URL: https://issues.apache.org/jira/browse/YARN-5858
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Varun Saxena
>            Assignee: Ajith S
>            Priority: Minor
>
> {noformat}
> java.lang.AssertionError: NodeManager could not identify disk failure.
> 	at org.junit.Assert.fail(Assert.java:88)
> 	at org.junit.Assert.assertTrue(Assert.java:41)
> 	at org.apache.hadoop.yarn.server.TestDiskFailures.verifyDisksHealth(TestDiskFailures.java:239)
> 	at org.apache.hadoop.yarn.server.TestDiskFailures.testDirsFailures(TestDiskFailures.java:202)
> 	at org.apache.hadoop.yarn.server.TestDiskFailures.testLogDirsFailures(TestDiskFailures.java:111)
> {noformat}
> Refer to https://builds.apache.org/job/PreCommit-YARN-Build/13828/testReport/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message