hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-512) Log aggregation root directory check is more expensive than it needs to be
Date Wed, 29 May 2013 23:16:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13669866#comment-13669866

Alejandro Abdelnur commented on YARN-512:

AFAIK the FileSystem caching does not have eviction by inactivity. Where do they get close
every 60sec?

> Log aggregation root directory check is more expensive than it needs to be
> --------------------------------------------------------------------------
>                 Key: YARN-512
>                 URL: https://issues.apache.org/jira/browse/YARN-512
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.0.5-beta
>            Reporter: Jason Lowe
>            Assignee: Maysam Yabandeh
>            Priority: Minor
>             Fix For: 2.0.5-beta
>         Attachments: YARN-512.patch
> The log aggregation root directory check first does an {{exists}} call followed by a
{{getFileStatus}} call.  That effectively stats the file twice.  It should just use {{getFileStatus}}
and catch {{FileNotFoundException}} to handle the non-existent case.
> In addition we may consider caching the presence of the directory rather than checking
it each time a node aggregates logs for an application.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message