tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "TezQA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-3990) The number of shuffle penalties for a host/inputAttemptIdentifier should be capped
Date Wed, 10 Oct 2018 19:19:00 GMT

    [ https://issues.apache.org/jira/browse/TEZ-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645440#comment-16645440
] 

TezQA commented on TEZ-3990:
----------------------------

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m  0s{color}
| {color:green} The patch appears to include 1 new or modified test files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 22s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 13s{color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 57s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 50s{color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 20s{color} |
{color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 14s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 12s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 12s{color} | {color:green}
the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  0m 26s{color}
| {color:orange} tez-runtime-library: The patch generated 9 new + 706 unchanged - 2 fixed
= 715 total (was 708) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  0s{color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 43s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 14s{color} |
{color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  4m  2s{color} | {color:green}
tez-runtime-library in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 10s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 59s{color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | TEZ-3990 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12943280/TEZ-3990.006.patch
|
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  checkstyle  compile
 |
| uname | Linux asf910.gq1.ygridcore.net 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28
UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/yetus/precommit/personality/tez.sh
|
| git revision | master / 2f17388 |
| maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
|
| Default Java | 1.8.0_172 |
| findbugs | v3.0.1 |
| checkstyle | https://builds.apache.org/job/PreCommit-TEZ-Build/23/artifact/out/diff-checkstyle-tez-runtime-library.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-TEZ-Build/23/testReport/ |
| modules | C: tez-runtime-library U: tez-runtime-library |
| Console output | https://builds.apache.org/job/PreCommit-TEZ-Build/23/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> The number of shuffle penalties for a host/inputAttemptIdentifier should be capped
> ----------------------------------------------------------------------------------
>
>                 Key: TEZ-3990
>                 URL: https://issues.apache.org/jira/browse/TEZ-3990
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.9.1, 0.10.0
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>            Priority: Major
>         Attachments: TEZ-3990.001.patch, TEZ-3990.002.patch, TEZ-3990.003.patch, TEZ-3990.004.patch,
TEZ-3990.005.patch, TEZ-3990.006.patch
>
>
> In a scenario where the same mapId fetches fail, the penalty code allows adding the same
Host/InputAttemptIdentifier over and over with revised penalty time that grows exponentially.
It should at some point drop the retrying and report failure to the AM asap to allow the job
to rectify the upstream output.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message