hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ádám Szita (Jira) <j...@apache.org>
Subject [jira] [Assigned] (HIVE-22460) LRFU cache policy leaks locked buffers upon purge()
Date Tue, 05 Nov 2019 10:47:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-22460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ádám Szita reassigned HIVE-22460:
---------------------------------


> LRFU cache policy leaks locked buffers upon purge()
> ---------------------------------------------------
>
>                 Key: HIVE-22460
>                 URL: https://issues.apache.org/jira/browse/HIVE-22460
>             Project: Hive
>          Issue Type: Bug
>          Components: llap
>            Reporter: Ádám Szita
>            Assignee: Ádám Szita
>            Priority: Major
>
> LRFU policy's purge() implementation is carefully not removing buffers that are currently
locked (i.e. in use by some IO thread). So far that's good.
> However it won't keep track of such buffers after the purge() method has finished: it
will always reset its heap and list, thereby forgetting information on these buffers. It will
never be able to evict these in the future, even if they get unlocked and become eligible
for eviction.
> This is problematic as:
>  * Although eventually these buffers might be evicted by BufferAllocator, by the time
that happens we have wasted space and time.
>  * Meta information about the buffers will remain in CacheContentsTracker forever, wasting
heap space too.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message