hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6547) Enhance SLS-based tests leveraging invariant checker
Date Sat, 13 May 2017 00:39:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008995#comment-16008995
] 

Wangda Tan commented on YARN-6547:
----------------------------------

[~curino],

I found a several issues once applied the patch, not sure if it is related to my environment:

Issue #1:
{code}
java.lang.ClassCastException: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics
cannot be cast to org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics

	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:223)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:191)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1298)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1384)
	at org.apache.hadoop.yarn.sls.scheduler.SLSFairScheduler.serviceInit(SLSFairScheduler.java:322)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:737)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1165)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:317)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.yarn.sls.SLSRunner.startRM(SLSRunner.java:226)
	at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:184)
	at org.apache.hadoop.yarn.sls.SLSRunner.run(SLSRunner.java:845)
	at org.apache.hadoop.yarn.sls.BaseSLSRunnerTest.runSLS(BaseSLSRunnerTest.java:116)
	at org.apache.hadoop.yarn.sls.TestSLSRunner.testSimulatorRunning(TestSLSRunner.java:86)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code} 

Issue #2: 
{code}
org.apache.hadoop.yarn.server.resourcemanager.monitor.invariants.InvariantViolationException:
Invariant "AppsSubmitted == 10" is NOT holding, with bindings: {AppsSubmitted=0}

	at org.apache.hadoop.yarn.server.resourcemanager.monitor.invariants.InvariantsChecker.logOrThrow(InvariantsChecker.java:74)
	at org.apache.hadoop.yarn.server.resourcemanager.monitor.invariants.MetricsInvariantChecker.editSchedule(MetricsInvariantChecker.java:175)
	at org.apache.hadoop.yarn.sls.BaseSLSRunnerTest.shutdownHookInvariantCheck(BaseSLSRunnerTest.java:144)
	at org.apache.hadoop.yarn.sls.BaseSLSRunnerTest.runSLS(BaseSLSRunnerTest.java:132)
	at org.apache.hadoop.yarn.sls.TestSLSRunner.testSimulatorRunning(TestSLSRunner.java:86)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code} 
This is RUMEN test, not sure if it is related to YARN-6111

Issue #3:
{code}
java.lang.NullPointerException
	at org.apache.hadoop.yarn.server.resourcemanager.monitor.invariants.MetricsInvariantChecker.init(MetricsInvariantChecker.java:93)
	at org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor.serviceInit(SchedulingMonitor.java:55)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:737)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1165)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:317)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.hadoop.yarn.sls.SLSRunner.startRM(SLSRunner.java:226)
	at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:184)
	at org.apache.hadoop.yarn.sls.SLSRunner.run(SLSRunner.java:845)
	at org.apache.hadoop.yarn.sls.BaseSLSRunnerTest.runSLS(BaseSLSRunnerTest.java:116)
	at org.apache.hadoop.yarn.sls.TestSLSRunner.testSimulatorRunning(TestSLSRunner.java:86)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}

In my local environment, all tests of TestSLSRunner cannot get passed. I'm running under OSX,
could you have chance to look at these problems?


> Enhance SLS-based tests leveraging invariant checker
> ----------------------------------------------------
>
>                 Key: YARN-6547
>                 URL: https://issues.apache.org/jira/browse/YARN-6547
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Carlo Curino
>            Assignee: Carlo Curino
>         Attachments: YARN-6547.v0.patch
>
>
> We can leverage {{InvariantChecker}}s to provide a more thorough validation of SLS-based
tests. This patch introduces invariants checking during and at the end of the run.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message