hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18375) The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them
Date Tue, 18 Jul 2017 10:01:06 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091367#comment-16091367

ramkrishna.s.vasudevan commented on HBASE-18375:

I think I get it now. Thanks for the explanation.
So here in removeChunk
  Chunk removeChunk(int chunkId) {
    WeakReference<Chunk> weak = this.weakChunkIdMap.remove(chunkId);
    Chunk strong = this.strongChunkIdMap.remove(chunkId);
    if (weak != null) {
      return weak.get();
    return strong;
So this chunk that we return from weak.get() at this point of time has a reference. But when
we are actually putting it to the reclaimedChunks ie.
Chunk chunk = ChunkCreator.this.removeChunk(chunkId);
        if (chunk != null) {
          if (chunk.isFromPool() && toAdd > 0) {
The chunk becomes null. So when we do poll from reclaimedchunks we get a null ref and we just
try to use that null ref and add it to the weakRefMap again and so we are working with null
reference through out as per
bq.And yes, it was still possible to poll the chunk from reclaimedChunks, but it was deallocated
while being in weakMap.
So in case of CellChunkMap atleast since we are sure to have a strong ref is it better to
always return the strong ref in removeChunk() code rather than the weak ref (though it is
Also the change in CompactionPipeline of removing and adding the last, now being done at the
beginning is it intentional to avoid the NPE bug?

> The pool chunks from ChunkCreator are deallocated while in pool because there is no reference
to them
> -----------------------------------------------------------------------------------------------------
>                 Key: HBASE-18375
>                 URL: https://issues.apache.org/jira/browse/HBASE-18375
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0-alpha-1
>            Reporter: Anastasia Braginsky
>            Priority: Critical
>             Fix For: 2.0.0, 3.0.0, 2.0.0-alpha-2
>         Attachments: HBASE-18375-V01.patch, HBASE-18375-V02.patch, HBASE-18375-V03.patch
> Because MSLAB list of chunks was changed to list of chunk IDs, the chunks returned back
to pool can be deallocated by JVM because there is no reference to them. The solution is to
protect pool chunks from GC by the strong map of ChunkCreator introduced by HBASE-18010. Will
prepare the patch today.

This message was sent by Atlassian JIRA

View raw message