carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CARBONDATA-375) Dictionary cache not getting cleared after task completion in dictionary decoder
Date Thu, 03 Nov 2016 16:01:59 GMT

    [ https://issues.apache.org/jira/browse/CARBONDATA-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15633257#comment-15633257
] 

ASF GitHub Bot commented on CARBONDATA-375:
-------------------------------------------

GitHub user manishgupta88 opened a pull request:

    https://github.com/apache/incubator-carbondata/pull/292

    [CARBONDATA-375] Dictionary cache not getting cleared after task completion in dictionary
decoder

    Problem: Dictionary cache not getting cleared after task completion in dictionary decoder
    
    Analysis: Currently LRU cache eviction policy is based on dictionary access count. For
cache to remove a entry its access count must be 0. In dictionary decoder after conversion
of surrogate key to actual value the access count for dictionary columns in query is not getting
decremented due to which it will never be cleared from memory when LRU cache size is configured.
    
    Fix: Add a task completion listener which will take care of clearing the dictionary in
case of both success and failure
    
    Impact area: LRU cache eviction policy which can lead to query and data load failure

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/manishgupta88/incubator-carbondata dictionary_decoder_clear_dictionary

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-carbondata/pull/292.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #292
    
----
commit b305f34e1014267b3706c287cef7070189fc3c28
Author: manishgupta88 <tomanishgupta18@gmail.com>
Date:   2016-11-03T15:48:03Z

    Problem: Dictionary cache not getting cleared after task completion in dictionary decoder
    
    Analysis: Currently LRU cache eviction policy is based on dictionary access count. For
cache to remove a entry its access count must be 0. In dictionary decoder after conversion
of surrogate key to actual value the access count for dictionary columns in query is not getting
decremented due to which it will never be cleared from memory when LRU cache size is configured.
    
    Fix: Add a task completion listener which will take care of clearing the dictionary in
case of both success and failure
    
    Impact area: LRU cache eviction policy which can lead to query and data load failure

----


> Dictionary cache not getting cleared after task completion in dictionary decoder
> --------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-375
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-375
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Manish Gupta
>            Assignee: Manish Gupta
>            Priority: Minor
>             Fix For: 0.2.0-incubating
>
>
> Currently LRU cache eviction policy is based on dictionary access count. For cache to
remove a entry its access count must be 0. In dictionary decoder after conversion of surrogate
key to actual value the access count for dictionary columns in query is not getting decremented
due to which it will never be cleared from memory when LRU cache size is configured.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message