tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kuhu Shukla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-3741) Tez outputs should free memory when closed
Date Thu, 01 Jun 2017 15:32:04 GMT

    [ https://issues.apache.org/jira/browse/TEZ-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033163#comment-16033163

Kuhu Shukla commented on TEZ-3741:

Thanks [~jlowe] for the updated patch!

I should have caught this sooner but may be we need a similar change for PipelinedSorter (SortSpan.close())
for completeness. Thoughts?

 boolean isClosed() {
    return kvbuffer == null && kvmeta == null;

Do we want it marked as @VisibleForTesting or have it available for other src/main files in
the package?

> Tez outputs should free memory when closed
> ------------------------------------------
>                 Key: TEZ-3741
>                 URL: https://issues.apache.org/jira/browse/TEZ-3741
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.7.1, 0.9.0
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: TEZ-3741.001.patch, TEZ-3741.002.patch
> Memory buffers aren't being released as quickly as they could be, e.g.: DefaultSorter
is holding onto the very large kvbuffer byte array even after close() is called, and Ordered
and Unordered outputs should remove references to sorter and kvWriter in their close.

This message was sent by Atlassian JIRA

View raw message