hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5003) AM recovery should recreate records for attempts that were incomplete
Date Thu, 29 Oct 2015 23:19:28 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981547#comment-14981547
] 

Hadoop QA commented on MAPREDUCE-5003:
--------------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s {color} | {color:blue}
docker + precommit patch detected. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color}
| {color:green} The patch appears to include 3 new or modified test files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 15s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 22s {color} |
{color:green} trunk passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 17s {color} |
{color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 20s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 26s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 46s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s {color} |
{color:green} trunk passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s {color} |
{color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 13s {color} | {color:red}
hadoop-mapreduce-client-app in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 15s {color} |
{color:green} the patch passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 15s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 22s {color} |
{color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 22s {color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 18s {color} | {color:red}
Patch generated 4 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client
(total was 604, now 605). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 26s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color}
| {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s {color} |
{color:green} the patch passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s {color} |
{color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 29s {color} | {color:red}
hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 36s {color} | {color:red}
hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 52s {color} | {color:red}
hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 52s {color} | {color:red}
hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 21s {color} | {color:red}
Patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 46s {color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| JDK v1.7.0_79 Failed junit tests | hadoop.mapreduce.v2.app.TestRecovery |
|   | hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler |
|   | hadoop.mapreduce.filecache.TestClientDistributedCacheManager |
|   | hadoop.mapreduce.v2.app.TestRecovery |
|   | hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler |
|   | hadoop.mapreduce.filecache.TestClientDistributedCacheManager |
| JDK v1.7.0_79 Timed out junit tests | org.apache.hadoop.mapreduce.v2.app.TestFetchFailure
|
|   | org.apache.hadoop.mapreduce.v2.app.TestMRApp |
|   | org.apache.hadoop.mapreduce.v2.app.TestFetchFailure |
|   | org.apache.hadoop.mapreduce.v2.app.TestMRApp |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-29 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12769629/MAPREDUCE-5003.9.patch
|
| JIRA Issue | MAPREDUCE-5003 |
| Optional Tests |  asflicense  javac  javadoc  mvninstall  unit  findbugs  checkstyle  compile
 |
| uname | Linux 3ae765e62ff0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-c3a2069/precommit/personality/hadoop.sh
|
| git revision | trunk / c293c58 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions |  /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
|
| findbugs | v3.0.0 |
| mvninstall | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
|
| checkstyle | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
|
| unit | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt
|
| unit | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
|
| unit | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt
|
| unit | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
|
| unit test logs |  https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
|
| JDK v1.7.0_79  Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/testReport/
|
| asflicense | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client
|
| Max memory used | 227MB |
| Powered by | Apache Yetus   http://yetus.apache.org |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6101/console |


This message was automatically generated.



> AM recovery should recreate records for attempts that were incomplete
> ---------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5003
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5003
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mr-am
>            Reporter: Jason Lowe
>            Assignee: Chang Li
>         Attachments: MAPREDUCE-5003.1.patch, MAPREDUCE-5003.2.patch, MAPREDUCE-5003.3.patch,
MAPREDUCE-5003.4.patch, MAPREDUCE-5003.5.patch, MAPREDUCE-5003.5.patch, MAPREDUCE-5003.6.patch,
MAPREDUCE-5003.7.patch, MAPREDUCE-5003.8.patch, MAPREDUCE-5003.9.patch, MAPREDUCE-5003.9.patch
>
>
> As discussed in MAPREDUCE-4992, it would be nice if the AM recovered task attempt entries
for *all* task attempts launched by the prior app attempt even if those task attempts did
not complete.  The attempts would have to be marked as killed or something similar to indicate
it is no longer running.  Having records for the task attempts enables the user to see what
nodes were associated with the attempts and potentially access their logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message