[ https://issues.apache.org/jira/browse/YARN-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-742:
----------------------------
Attachment: YARN-742.patch
Patch to walk back up the app log dir path to check for the existence of a directory before
blindly proceeding to create it. It gets out early if it finds a path that exists.
In addition to the unit test, I manually tested this on a single-node cluster verifying the
directories are created iff they are missing and permissions are set iff necessary due to
a too-restrictive umask.
> Log aggregation causes a lot of redundant setPermission calls
> -------------------------------------------------------------
>
> Key: YARN-742
> URL: https://issues.apache.org/jira/browse/YARN-742
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.7, 2.0.4-alpha
> Reporter: Kihwal Lee
> Assignee: Jason Lowe
> Attachments: YARN-742.patch
>
>
> In one of our clusters, namenode RPC is spending 45% of its time on serving setPermission
calls. Further investigation has revealed that most calls are redundantly made on /mapred/logs/<user>/logs.
Also mkdirs calls are made before this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
|