hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anastasia Braginsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18375) The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them
Date Tue, 08 Aug 2017 14:03:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118358#comment-16118358
] 

Anastasia Braginsky commented on HBASE-18375:
---------------------------------------------

Hey [~anoop.hbase],

Great minds thinks alike!
We have just realized what the problem is here (in details) and I came to write the comment
and saw your comment :)
As [~ram_krish] has written we didn't know the true reason why this patch resolves the problem,
so I continued to look for the "truth".

So now we have the full understanding what is going on. Here it is:
1. The Maps in ChunkCreator are referencing chunks, while Cells in CSLM and CellArayMap are
referencing ByteBuffers. 
2. Therefore chunks that are referenced from WeakMap or SoftMap are getting released, however
ByteBuffers are still there.
3. There is no meaning in WeakMap or SoftMap. Only StrongMap can truly save the chunks from
GC while they are still in active CSLM. So only StrongMap really helps for CellChunkMap.

>From here the patch presented here resolves a true problem of pool chunks being not returned
from pool.
The problem exists also in Release 2.0 (!), but since the references in map there are soft
references, which are released last, the frequency of problem to appear is less.

As solution I suggest to use only StrongMap for all chunks. So let me present another patch
for this JIRA.
As solution to Release 2.0 I would suggest to bring back the chunks (from pool) linked list
into the MSLAB instead of chunk IDs linked list.
What do you think?
bq. Are we always sure when the above is called, the weak map will have refs to Chunks still
?
Clearly not, the chunks are released very fast, just after they were added to CSLM.


> The pool chunks from ChunkCreator are deallocated while in pool because there is no reference
to them
> -----------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-18375
>                 URL: https://issues.apache.org/jira/browse/HBASE-18375
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0-alpha-1
>            Reporter: Anastasia Braginsky
>            Assignee: Anastasia Braginsky
>            Priority: Critical
>             Fix For: 2.0.0, 3.0.0, 2.0.0-alpha-2
>
>         Attachments: HBASE-18375-V01.patch, HBASE-18375-V02.patch, HBASE-18375-V03.patch,
HBASE-18375-V04.patch
>
>
> Because MSLAB list of chunks was changed to list of chunk IDs, the chunks returned back
to pool can be deallocated by JVM because there is no reference to them. The solution is to
protect pool chunks from GC by the strong map of ChunkCreator introduced by HBASE-18010. Will
prepare the patch today.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message