tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "TezQA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-3990) The number of shuffle penalties for a host/inputAttemptIdentifier should be capped
Date Wed, 10 Oct 2018 18:17:00 GMT

    [ https://issues.apache.org/jira/browse/TEZ-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645369#comment-16645369
] 

TezQA commented on TEZ-3990:
----------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m  0s{color}
| {color:green} The patch appears to include 1 new or modified test files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m  4s{color} | {color:red}
tez-runtime-library in master failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m  5s{color} | {color:red}
tez-runtime-library in master failed with JDK v1.8.0_181. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m  5s{color} | {color:red}
tez-runtime-library in master failed with JDK v1.8.0_172. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 26s{color}
| {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m  4s{color} | {color:red}
tez-runtime-library in master failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  4s{color} | {color:red}
tez-runtime-library in master failed with JDK v1.8.0_181. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  4s{color} | {color:red}
tez-runtime-library in master failed with JDK v1.8.0_172. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m  5s{color} | {color:red}
tez-runtime-library in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m  5s{color} | {color:red}
tez-runtime-library in the patch failed with JDK v1.8.0_181. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m  5s{color} | {color:red}
tez-runtime-library in the patch failed with JDK v1.8.0_181. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m  5s{color} | {color:red}
tez-runtime-library in the patch failed with JDK v1.8.0_172. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m  5s{color} | {color:red}
tez-runtime-library in the patch failed with JDK v1.8.0_172. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  0m 26s{color}
| {color:orange} tez-runtime-library: The patch generated 9 new + 707 unchanged - 2 fixed
= 716 total (was 709) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  0s{color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m  4s{color} | {color:red}
tez-runtime-library in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  5s{color} | {color:red}
tez-runtime-library in the patch failed with JDK v1.8.0_181. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  5s{color} | {color:red}
tez-runtime-library in the patch failed with JDK v1.8.0_172. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  0m  4s{color} | {color:red}
tez-runtime-library in the patch failed with JDK v1.8.0_172. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m  4s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}  2m 16s{color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | TEZ-3990 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12943280/TEZ-3990.006.patch
|
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  checkstyle  compile
 |
| uname | Linux asf912.gq1.ygridcore.net 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43
UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/yetus/precommit/personality/tez.sh
|
| git revision | master / 2f17388 |
| maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
|
| Default Java | 1.8.0_172 |
| Multi-JDK versions |  /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_181 /usr/local/asfpackages/java/jdk1.8.0_172:1.8.0_172
|
| mvninstall | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/branch-mvninstall-tez-runtime-library.txt
|
| compile | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/branch-compile-tez-runtime-library-jdk1.8.0_181.txt
|
| compile | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/branch-compile-tez-runtime-library-jdk1.8.0_172.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/branch-findbugs-tez-runtime-library.txt
|
| javadoc | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/branch-javadoc-tez-runtime-library-jdk1.8.0_181.txt
|
| javadoc | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/branch-javadoc-tez-runtime-library-jdk1.8.0_172.txt
|
| mvninstall | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/patch-mvninstall-tez-runtime-library.txt
|
| compile | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/patch-compile-tez-runtime-library-jdk1.8.0_181.txt
|
| javac | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/patch-compile-tez-runtime-library-jdk1.8.0_181.txt
|
| compile | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/patch-compile-tez-runtime-library-jdk1.8.0_172.txt
|
| javac | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/patch-compile-tez-runtime-library-jdk1.8.0_172.txt
|
| checkstyle | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/diff-checkstyle-tez-runtime-library.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/patch-findbugs-tez-runtime-library.txt
|
| javadoc | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/patch-javadoc-tez-runtime-library-jdk1.8.0_181.txt
|
| javadoc | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/patch-javadoc-tez-runtime-library-jdk1.8.0_172.txt
|
| unit | https://builds.apache.org/job/PreCommit-TEZ-Build/22/artifact/out/patch-unit-tez-runtime-library-jdk1.8.0_172.txt
|
| JDK v1.8.0_172  Test Results | https://builds.apache.org/job/PreCommit-TEZ-Build/22/testReport/
|
| modules | C: tez-runtime-library U: tez-runtime-library |
| Console output | https://builds.apache.org/job/PreCommit-TEZ-Build/22/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> The number of shuffle penalties for a host/inputAttemptIdentifier should be capped
> ----------------------------------------------------------------------------------
>
>                 Key: TEZ-3990
>                 URL: https://issues.apache.org/jira/browse/TEZ-3990
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.9.1, 0.10.0
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>            Priority: Major
>         Attachments: TEZ-3990.001.patch, TEZ-3990.002.patch, TEZ-3990.003.patch, TEZ-3990.004.patch,
TEZ-3990.005.patch, TEZ-3990.006.patch
>
>
> In a scenario where the same mapId fetches fail, the penalty code allows adding the same
Host/InputAttemptIdentifier over and over with revised penalty time that grows exponentially.
It should at some point drop the retrying and report failure to the AM asap to allow the job
to rectify the upstream output.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message