hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kuhu Shukla (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6315) Improve LocalResourcesTrackerImpl#isResourcePresent to return false for corrupted files
Date Sun, 12 Mar 2017 18:02:04 GMT

     [ https://issues.apache.org/jira/browse/YARN-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Kuhu Shukla updated YARN-6315:
    Attachment: YARN-6315.001.patch

First version of the patch that uses readAttributes bulk operation to match the size for resources
that are not directories since the size of the directory may not always match up. It maintains
the exists() behavior by setting ret= false when file not found exception is thrown. The method
also catches IOException to maintain previous behavior/signature. 

> Improve LocalResourcesTrackerImpl#isResourcePresent to return false for corrupted files
> ---------------------------------------------------------------------------------------
>                 Key: YARN-6315
>                 URL: https://issues.apache.org/jira/browse/YARN-6315
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.3, 2.8.1
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>         Attachments: YARN-6315.001.patch
> We currently check if a resource is present by making sure that the file exists locally.
There can be a case where the LocalizationTracker thinks that it has the resource if the file
exists but with size 0 or less than the "expected" size of the LocalResource. This JIRA tracks
the change to harden the isResourcePresent call to address that case.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message