hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tarun Parimi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-9040) LevelDBCacheTimelineStore in ATS 1.5 leaks native memory
Date Thu, 29 Nov 2018 08:38:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702860#comment-16702860
] 

Tarun Parimi commented on YARN-9040:
------------------------------------

[~rohithsharma] Can you please review this ?

> LevelDBCacheTimelineStore in ATS 1.5 leaks native memory
> --------------------------------------------------------
>
>                 Key: YARN-9040
>                 URL: https://issues.apache.org/jira/browse/YARN-9040
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: timelineserver
>    Affects Versions: 2.8.0
>            Reporter: Tarun Parimi
>            Assignee: Tarun Parimi
>            Priority: Major
>         Attachments: YARN-9040.001.patch, YARN-9040.002.patch
>
>
> When LevelDBCacheTimelineStore from YARN-4219 is used as ATS 1.5 entity caching storage, we
observe memory leak due to leveldb files even after the fix of YARN-5368 .
> Top output shows 0.024TB (25GB) RES, even though heap size is only 8GB.
>  
>  
> {code:java}
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 25519 yarn 20 0 33.024g 0.024t 41468 S 6.2 26.0 21:07.39 /usr/java/default/bin/java -Dproc_timelineserver
-Xmx8192m
> {code}
>  
> Lsof shows a lot of open timeline-cache.ldb files which are referenced by ATS, even
though are deleted (DEL), since they are not present when listing them .
>  
> {code:java}
> java 25519 yarn DEL REG 253,28 9438452 /var/yarn/timeline/timelineEntityGroupId_1542280269959_55569_dag_1542280269959_55569_2-timeline-cache.ldb/000007.sst
> java 25519 yarn DEL REG 253,28 9438438 /var/yarn/timeline/timelineEntityGroupId_1542280269959_55569_dag_1542280269959_55569_2-timeline-cache.ldb/000007.sst
> java 25519 yarn DEL REG 253,28 9438437 /var/yarn/timeline/timelineEntityGroupId_1542280269959_55569_dag_1542280269959_55569_2-timeline-cache.ldb/000005.sst
> {code}
>  
> Looks like LevelDBCacheTimelineStore is not closing these files as the LevelDB DBIterator
is not closed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message