hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhihai xu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-6396) Call verifyAndCreateRemoteLogDir at service initialization instead of application initialization to decrease load for name node
Date Mon, 27 Mar 2017 04:58:41 GMT
zhihai xu created YARN-6396:
-------------------------------

             Summary: Call verifyAndCreateRemoteLogDir at service initialization instead of
application initialization to decrease load for name node
                 Key: YARN-6396
                 URL: https://issues.apache.org/jira/browse/YARN-6396
             Project: Hadoop YARN
          Issue Type: Improvement
          Components: log-aggregation
    Affects Versions: 3.0.0-alpha2
            Reporter: zhihai xu
            Assignee: zhihai xu
            Priority: Minor


Call verifyAndCreateRemoteLogDir at service initialization instead of application initialization
to decrease load for name node.
Currently for every application at each Node, verifyAndCreateRemoteLogDir will be called before
doing log aggregation, This will be a non trivial overhead for name node in a large cluster
since verifyAndCreateRemoteLogDir calls getFileStatus. Once the remote log directory is created
successfully, it is not necessary to call it again. It will be better to call verifyAndCreateRemoteLogDir
at LogAggregationService service initialization.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message