hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15101) Leaked References to StoreFile.Reader after HBASE-13082
Date Tue, 19 Jan 2016 07:44:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15106396#comment-15106396

ramkrishna.s.vasudevan commented on HBASE-15101:

Patch looks good to me. ya I think the shorter version is the easiest way after seeing the
code where the patch is prepared.
Regarding the point where you say a Region Server dies and you end up seeing lot of uncollected
files, in the version without HBASE-13082 - suppose a set of compactions started but that
RS got killed or dies before it is completed you wil have all those store files again getting
used for subsequent readers when the regions are opened in a new RS and there should be another
compaction that needs to run and move the files to the archive dir. Now even after HBASE-13082
- the scenario is similar because once the compaction is done if the Compaction discharger
thread does not kick in and before that your RS dies, you end up with the store files again
available for reads and you need one more set of compaction to happen in the new RS to move
the files to the archive dir. 
After applying the patch also you still get issues where the compacted files are not cleared?
 You mean there is always a set of files that is not cleared ?
Now coming to do the cleaning operation during close(), I think we are doing it on HStore.close()
operation and ensuring that the compacted files are moved to the archive dir. Its good to
check this out before we commit the updated patch to branch-1. 

> Leaked References to StoreFile.Reader after HBASE-13082
> -------------------------------------------------------
>                 Key: HBASE-15101
>                 URL: https://issues.apache.org/jira/browse/HBASE-15101
>             Project: HBase
>          Issue Type: Bug
>          Components: HFile, io
>    Affects Versions: 2.0.0
>            Reporter: deepankar
>            Assignee: deepankar
>         Attachments: HBASE-15101-v1.patch, HBASE-15101-v2.patch, HBASE-15101-v3.patch,
> We observed this production that after a region server dies there are huge number of
hfiles in that region for the region server running the version with HBASE-13082, In the doc
it is given that it is expected to happen, but we found a one place where scanners are not
being closed. If the scanners are not closed their references are not decremented and that
is leading to the issue of huge number of store files not being finalized
> All I was able to find is in the selectScannersFrom, where we discard some of the scanners
and we are not closing them. I am attaching a patch for that.
> Also to avoid these issues should the files that are done be logged and finalized (moved
to archive) as a part of region close operation. This will solve any leaks that can happen
and does not cause any dire consequences?

This message was sent by Atlassian JIRA

View raw message