hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-574) PrivateLocalizer does not support parallel resource download via ContainerLocalizer
Date Fri, 21 Apr 2017 20:29:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979330#comment-15979330
] 

Hadoop QA commented on YARN-574:
--------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  0s{color} | {color:blue}
Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  7s{color} | {color:red}
YARN-574 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-574 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12849264/YARN-574.05.patch
|
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/15712/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> PrivateLocalizer does not support parallel resource download via ContainerLocalizer
> -----------------------------------------------------------------------------------
>
>                 Key: YARN-574
>                 URL: https://issues.apache.org/jira/browse/YARN-574
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 2.6.0, 2.8.0, 2.7.1
>            Reporter: Omkar Vinit Joshi
>            Assignee: Ajith S
>         Attachments: YARN-574.03.patch, YARN-574.04.patch, YARN-574.05.patch, YARN-574.1.patch,
YARN-574.2.patch
>
>
> At present private resources will be downloaded in parallel only if multiple containers
request the same resource. However otherwise it will be serial. The protocol between PrivateLocalizer
and ContainerLocalizer supports multiple downloads however it is not used and only one resource
is sent for downloading at a time.
> I think we can increase / assure parallelism (even for single container requesting resource)
for private/application resources by making multiple downloads per ContainerLocalizer.
> Total Parallelism before
> = number of threads allotted for PublicLocalizer [public resource] + number of containers[private
and application resource]
> Total Parallelism after
> = number of threads allotted for PublicLocalizer [public resource] + number of containers
* max downloads per container [private and application resource]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message