apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "bright chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2321) Improve Buckets memory management
Date Tue, 01 Nov 2016 19:01:00 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15626347#comment-15626347

bright chen commented on APEXMALHAR-2321:

I changed to use keyStream and valueStream to manage the memory. But as we can't enforce the
caller to use keyStream and valueStream to manage the memory, So it is also need to cleanup
the maps.

[~csingh] How do you think? and I saw you also clear the fileCache and close readers in freeMemory().
Is it better to move to another function or rename this method into something like releaseResource?

> Improve Buckets memory management
> ---------------------------------
>                 Key: APEXMALHAR-2321
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2321
>             Project: Apache Apex Malhar
>          Issue Type: Improvement
>            Reporter: bright chen
>            Assignee: bright chen
> Currently buckets were managed as an array. Each bucket have memory limitation, and free
memory will be triggered if the bucket memory usage over the limitation.
> - For ManagedTimeUnifiedStateImpl, the default bucket number is 345600, which probably
too large. But it can be changed by set Context.OperatorContext.APPLICATION_WINDOW_COUNT
> - The default maxMemorySize is zero. It's better to give a default reasonable value to
avoid too much garbage collection 

This message was sent by Atlassian JIRA

View raw message