hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8867) Retrieve the progress of resource localization
Date Tue, 30 Oct 2018 20:59:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669325#comment-16669325
] 

Eric Yang commented on YARN-8867:
---------------------------------

[~csingh] The current wip patch looks ok for adding new communication protocol in yarn, node
manager and mapreduce client to retrieve localization status.  We probably want to define
what constitutes a localization status.  In CLI and UI, progress field seems to indicate a
numeric number of percentage.  The current communication protocol is a list of localization
status.  If we have container image to be localized, and it is represented by one of the localization
status.  It does not translate well to percentage values.  Docker image download is also hard
to be represented as percentage value because docker pull output does not show percentage
output.  Given those limitations from backend, it would be in our best interests to define
what user will see as progression.  If progress percentage is still the information that is
going to be displayed, we may want to spend more time on how to calculate status to progress
percentage.  Otherwise, localizing a single container without other resource might look odd
for UI to stuck on 0 percent and jump to 100 percent because there is only one localization
status.

> Retrieve the progress of resource localization
> ----------------------------------------------
>
>                 Key: YARN-8867
>                 URL: https://issues.apache.org/jira/browse/YARN-8867
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Chandni Singh
>            Assignee: Chandni Singh
>            Priority: Major
>         Attachments: YARN-8867.wip.patch
>
>
> Refer YARN-3854.
> Currently NM does not have an API to retrieve the progress of localization. Unless the
client can know when the localization of a resource is complete irrespective of the type of
the resource, it cannot take any appropriate action. 
> We need an API in {{ContainerManagementProtocol}} to retrieve the progress on the localization.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message