Mailing-List: contact dev-help@apex.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@apex.apache.org
Date: Mon, 14 Nov 2016 04:44:58 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: dev@apex.incubator.apache.org
Message-ID: <JIRA.13019622.1478720255000.265756.1479098698396@Atlassian.JIRA>
In-Reply-To: <JIRA.13019622.1478720255000@Atlassian.JIRA>
References: <JIRA.13019622.1478720255000@Atlassian.JIRA> <JIRA.13019622.1478720255595@arcas>
Subject: [jira] [Commented] (APEXMALHAR-2335) Problems on StateTracker
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 14 Nov 2016 04:46:16 -0000


    [ https://issues.apache.org/jira/browse/APEXMALHAR-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15662751#comment-15662751 ] 

ASF GitHub Bot commented on APEXMALHAR-2335:
--------------------------------------------

Github user brightchen closed the pull request at:

    https://github.com/apache/apex-malhar/pull/493


> Problems on StateTracker
> ------------------------
>
>                 Key: APEXMALHAR-2335
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2335
>             Project: Apache Apex Malhar
>          Issue Type: Bug
>            Reporter: bright chen
>            Assignee: bright chen
>   Original Estimate: 144h
>  Remaining Estimate: 144h
>
> The intension of StateTracker#bucketAccessed as I understand was to keep the last access time of each bucket so freeMemory can free the buckets which not accessed recently.
> But here are some problems
> - StateTracker#bucketAccessed() will be called each time access to the bucket. Current implementation use Concurrent map and set to handle it(bucketHeap and bucketAccessTimes). it could be pretty heavy.
> - StateTracker#run use bucketHeap to get the bucket which need to free memory, but at the same time bucketHeap was constantly changed by bucketAccessed(). It could cause concurrent issue event bucketHeap is a Current set. And even more the while loop could never end. Following two bug probably related to this:
> https://issues.apache.org/jira/browse/APEXMALHAR-2333
> https://issues.apache.org/jira/browse/APEXMALHAR-2334
> Following are my thought to solve these issue.
> - It's not necessary to keep very accurate of update time, it probably only need to be updated once for each stream window. But StateTracker don't support endWindow() (in fact endWindow() was triggered at end of application window). So, we can keep the bucket which updated in this period and update only at the end of the period.
> - keep two sets of bucketHeap, one for bucketAccessed() and one for run(). so the lock only need when switch


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)