hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4334) Ability to avoid ResourceManager recovery if state store is "too old"
Date Fri, 20 Nov 2015 01:56:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015025#comment-15015025
] 

Hadoop QA commented on YARN-4334:
---------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 9s {color} | {color:blue}
docker + precommit patch detected. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color}
| {color:green} The patch appears to include 1 new or modified test files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 4s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 39s {color} |
{color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 36s {color} |
{color:green} trunk passed with JDK v1.7.0_85 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 32s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 55s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 49s {color}
| {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 34s {color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in trunk has 3 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 55s {color} |
{color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 12s {color} |
{color:green} trunk passed with JDK v1.7.0_85 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 49s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 35s {color} |
{color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 35s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 35s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 36s {color} |
{color:green} the patch passed with JDK v1.7.0_85 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 36s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 36s {color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 32s {color} | {color:red}
Patch generated 24 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 601,
now 622). {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 58s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 47s {color}
| {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red}
The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s {color} | {color:green}
The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 39s {color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager introduced
2 new FindBugs issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 52s {color} |
{color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 14s {color} |
{color:green} the patch passed with JDK v1.7.0_85 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 31s {color} | {color:red}
hadoop-yarn-api in the patch failed with JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 28s {color} | {color:green}
hadoop-yarn-common in the patch passed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 68m 51s {color} | {color:red}
hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 33s {color} | {color:red}
hadoop-yarn-api in the patch failed with JDK v1.7.0_85. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 28s {color} | {color:green}
hadoop-yarn-common in the patch passed with JDK v1.7.0_85. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 35s {color} | {color:red}
hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 27s {color}
| {color:green} Patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 196m 38s {color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
|  |  Switch statement found in org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.rememberTargetTransitionsAndStoreState(RMAppEvent,
Object, RMAppState, RMAppState) where default case is missing  At RMAppImpl.java:Object, RMAppState,
RMAppState) where default case is missing  At RMAppImpl.java:[lines 1114-1126] |
|  |  Switch statement found in org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.rememberTargetTransitionsAndStoreState(RMAppAttemptEvent,
Object, RMAppAttemptState, RMAppAttemptState) where default case is missing  At RMAppAttemptImpl.java:Object,
RMAppAttemptState, RMAppAttemptState) where default case is missing  At RMAppAttemptImpl.java:[lines
1220-1251] |
| JDK v1.8.0_66 Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
|   | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
|   | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
|   | hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore |
| JDK v1.7.0_85 Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
|   | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
|   | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
|   | hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler |
|   | hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:date2015-11-19 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12773355/YARN-4334.patch
|
| JIRA Issue | YARN-4334 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  findbugs
 checkstyle  xml  cc  |
| uname | Linux b4e36e806b63 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/apache-yetus-3f4279a/precommit/personality/hadoop.sh
|
| git revision | trunk / 31dc5c6 |
| findbugs | v3.0.0 |
| findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-warnings.html
|
| checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/whitespace-eol.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
|
| unit test logs |  https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt
https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt
https://builds.apache.org/job/PreCommit-YARN-Build/9743/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
|
| JDK v1.7.0_85  Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/9743/testReport/
|
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
|
| Max memory used | 78MB |
| Powered by | Apache Yetus   http://yetus.apache.org |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9743/console |


This message was automatically generated.



> Ability to avoid ResourceManager recovery if state store is "too old"
> ---------------------------------------------------------------------
>
>                 Key: YARN-4334
>                 URL: https://issues.apache.org/jira/browse/YARN-4334
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>            Reporter: Jason Lowe
>            Assignee: Chang Li
>         Attachments: YARN-4334.patch, YARN-4334.wip.2.patch, YARN-4334.wip.3.patch, YARN-4334.wip.4.patch,
YARN-4334.wip.patch
>
>
> There are times when a ResourceManager has been down long enough that ApplicationMasters
and potentially external client-side monitoring mechanisms have given up completely.  If the
ResourceManager starts back up and tries to recover we can get into situations where the RM
launches new application attempts for the AMs that gave up, but then the client _also_ launches
another instance of the app because it assumed everything was dead.
> It would be nice if the RM could be optionally configured to avoid trying to recover
if the state store was "too old."  The RM would come up without any applications recovered,
but we would avoid a double-submission situation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message