hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anastasia Braginsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18375) The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them
Date Wed, 19 Jul 2017 07:55:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092737#comment-16092737

Anastasia Braginsky commented on HBASE-18375:

Hey Ram!

Thanks for your questions, they make me rethink the process once again. 

I have not yet presented in this JIRA one more important problem introduced in HBASE-18010,
where the flattening became a creation pattern. The pipeline version wasn't increased while
flattening, so it allowed simultaneous change of the pipeline from the snapshot and from the
flattening. I wanted to add this fix in other JIRA, but as I see it is all related, I will
add this fix also here and I will publish a new patch soon.

bq. So this chunk that we return from weak.get() at this point of time has a reference.
If you are asking about the exact scenario happening in my failure, there the reference is
removed from strongMap. But generally poolChunk can be also removed from weakMap. And yes
the pool chunk still has the reference at this point in time ( weak.get() returns strong reference).

bq. But when we are actually putting it to the reclaimedChunks, the chunk becomes null. 
In my scenario, I believe the pool chunk still has reference also there and it is added to
reclaimedChunks in its "correct" form. But again if chunk is null then, and we don't add it
back to reclaimedChunks, it is still not a desired behavior, because we deallocate chunks
from pool and decreasing their number...

bq. So when we do poll from reclaimedchunks we get a null ref and we just try to use that
null ref and add it to the weakRefMap again and so we are working with null reference through
out as per
Again, I see the chunk is polled from reclaimedChunks when it still OK, but released later
when it is already in weakRefMap. It is a subtle scenario, but causing a problem after all.
To always keep pool chunks in strongMap is much cleaner and thus safer.

bq. So in case of CellChunkMap atleast since we are sure to have a strong ref is it better
to always return the strong ref in removeChunk() code rather than the weak ref (though it
is available)?
Sorry, I do not see why it is better to always return the strong ref in removeChunk() ...
? Currently removeChunk() returns strong reference anyway.

> The pool chunks from ChunkCreator are deallocated while in pool because there is no reference
to them
> -----------------------------------------------------------------------------------------------------
>                 Key: HBASE-18375
>                 URL: https://issues.apache.org/jira/browse/HBASE-18375
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0-alpha-1
>            Reporter: Anastasia Braginsky
>            Priority: Critical
>             Fix For: 2.0.0, 3.0.0, 2.0.0-alpha-2
>         Attachments: HBASE-18375-V01.patch, HBASE-18375-V02.patch, HBASE-18375-V03.patch
> Because MSLAB list of chunks was changed to list of chunk IDs, the chunks returned back
to pool can be deallocated by JVM because there is no reference to them. The solution is to
protect pool chunks from GC by the strong map of ChunkCreator introduced by HBASE-18010. Will
prepare the patch today.

This message was sent by Atlassian JIRA

View raw message