hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Omkar Vinit Joshi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-539) LocalizedResources are leaked in memory in case resource localization fails
Date Wed, 10 Apr 2013 18:59:16 GMT

    [ https://issues.apache.org/jira/browse/YARN-539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628115#comment-13628115
] 

Omkar Vinit Joshi commented on YARN-539:
----------------------------------------

The modified flow for Successful as well as Failed resource is
* Failed Resource download :- Public/Private localizer will notify tracker. Tracker removes
the resource from its cache (No memory leak now).  Then passes the event to LocalizedResource.
Resource send ContainerResourceFailedEvent to all the waiting containers. Containers in turn
send ResourceReleaseEvent. Earlier we thought about removing this Release call but it is required
as multiple resources requested by the container may fail one after the another before container's
release event is handled on all the requested resources due to one of the resource failure.
* Successful Resource download :- Public/Private localizer will notify tracker which in turn
will notify LocalizedResource. Resource informs all the Container of the successful download.
* Added Test TestLocalResourcesTrackerImpl.testLocalResourceCache for testing resource lifecycle
and memory leak
** 2 Containers are requesting the resource. After resource failure the containers are informed
and resource is removed from cache. Now before last container's ResourceReleaseEvent is handled
another container requests for the same resource. So the ResourceReleaseEvent will return
silently without exception. In the end after successful resource localization (for second
attempt) and ResourceReleasEvent (by container-3) resource remains in cache in LOCALIZED state
with zero containers in waiting queue.
                
> LocalizedResources are leaked in memory in case resource localization fails
> ---------------------------------------------------------------------------
>
>                 Key: YARN-539
>                 URL: https://issues.apache.org/jira/browse/YARN-539
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Omkar Vinit Joshi
>            Assignee: Omkar Vinit Joshi
>         Attachments: yarn-539-20130410.patch
>
>
> If resource localization fails then resource remains in memory and is
> 1) Either cleaned up when next time cache cleanup runs and there is space crunch. (If
sufficient space in cache is available then it will remain in memory).
> 2) reused if LocalizationRequest comes again for the same resource.
> I think when resource localization fails then that event should be sent to LocalResourceTracker
which will then remove it from its cache.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message