hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "yunjiong zhao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6339) Improve performance for createAndGetApplicationReport
Date Thu, 16 Mar 2017 22:25:41 GMT

     [ https://issues.apache.org/jira/browse/YARN-6339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

yunjiong zhao updated YARN-6339:
    Attachment: YARN-6339.002.patch

Update patch for more improvement.
Change RMAppImpl.logAggregationStatus from HashMap to ConcurrentHashMap so even hold a readlock,
we can safely update logAggregationStatus.
Then return Collections.unmodifiableMap to avoid create too many HashMap.

> Improve performance for createAndGetApplicationReport
> -----------------------------------------------------
>                 Key: YARN-6339
>                 URL: https://issues.apache.org/jira/browse/YARN-6339
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: yunjiong zhao
>            Assignee: yunjiong zhao
>         Attachments: YARN-6339.001.patch, YARN-6339.002.patch
> There are two performance issue when calling createAndGetApplicationReport:
> One is inside ProtoUtils.convertFromProtoFormat, replace is too slow for clusters which
have more than 3000 nodes. Use substring is much better: https://issues.apache.org/jira/browse/YARN-6285?focusedCommentId=15923241&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15923241
> Another one is inside getLogAggregationReportsForApp, if some application's LogAggregationStatus
is TIME_OUT, every time it was called it will create an HashMap which will produce lots of

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message