Return-Path: 
 <issues-return-223706-apmail-hbase-issues-archive=hbase.apache.org@hbase.apache.org>
X-Original-To: apmail-hbase-issues-archive@www.apache.org
Delivered-To: apmail-hbase-issues-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 86D16186B0
	for <apmail-hbase-issues-archive@www.apache.org>;
 Fri, 11 Dec 2015 09:05:11 +0000 (UTC)
Received: (qmail 57896 invoked by uid 500); 11 Dec 2015 09:05:11 -0000
Delivered-To: apmail-hbase-issues-archive@hbase.apache.org
Received: (qmail 57845 invoked by uid 500); 11 Dec 2015 09:05:11 -0000
Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@hbase.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@hbase.apache.org>
List-Post: <mailto:issues@hbase.apache.org>
List-Id: <issues.hbase.apache.org>
Delivered-To: mailing list issues@hbase.apache.org
Received: (qmail 57824 invoked by uid 99); 11 Dec 2015 09:05:11 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Dec 2015 09:05:11 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 04ABB2C1F5A
	for <issues@hbase.apache.org>; Fri, 11 Dec 2015 09:05:11 +0000 (UTC)
Date: Fri, 11 Dec 2015 09:05:11 +0000 (UTC)
From: "Anoop Sam John (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12916902.1448874495000.334134.1449824711016@Atlassian.JIRA>
In-Reply-To: <JIRA.12916902.1448874495000@Atlassian.JIRA>
References: <JIRA.12916902.1448874495000@Atlassian.JIRA>
 <JIRA.12916902.1448874495810@arcas>
Subject: [jira] [Commented] (HBASE-14895) Seek only to the newly flushed
 file on scanner reset on flush
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/HBASE-14895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15052479#comment-15052479 ] 

Anoop Sam John commented on HBASE-14895:
----------------------------------------

one issue found wrt maintain the scanners in a new List in StoreScanner (These are memstore scanner and store file scanners at the begin of the scan)
Over the scan time, some of the scanners would have reached its end and we would have eventually closed them off. And the PriorityQ in KVHeap would have removed them as well.  But we are still holding those refs in this new List in Store Scanner.
Now after a flush we will use this new List to recreate new Heap. We will remove old MemstoreScaner and add new MemstoreScanner as well as scanner to the new flushed file. And use all of them to create new KVHeap. (This might include already closed StoreFileScanners as well).
In KVHeap constructor 
{code}
KeyValueHeap(List<? extends KeyValueScanner> scanners,
      KVScannerComparator comparator) throws IOException {
    this.comparator = comparator;
    if (!scanners.isEmpty()) {
      this.heap = new PriorityQueue<KeyValueScanner>(scanners.size(),
          this.comparator);
      for (KeyValueScanner scanner : scanners) {
        if (scanner.peek() != null) {
          this.heap.add(scanner);
        } else {
          this.scannersForDelayedClose.add(scanner);
        }
      }
      this.current = pollRealKV();
    }
  }
  {code}
So the closed scanners wil have peek Cell as null we wont add them to heap. Good. But we will add them to delayed close and after one shipped call we will try close them again
StoreFileScanner#close()
{code}
public void close() {
    cur = null;
    this.hfs.close();
    if (this.reader != null) {
      this.reader.decrementRefCount();
    }
  }
  {code}
No impact because of hfs close. But the decr read ref count will be a problem and can cause issue.

In a simple way we can we can have closed flag in StoreFileScanner which can be used to check when close call on second time or close call nullifying the reader and hfs on StoreFileScanner.

Or else if we dont need to maintain a new List of scanners in StoreScanner and make use of the current scanners in the KVHeap for recreating the new heap, that would have looked better. No need to maintain same scanners in 2 data structures.


> Seek only to the newly flushed file on scanner reset on flush
> -------------------------------------------------------------
>
>                 Key: HBASE-14895
>                 URL: https://issues.apache.org/jira/browse/HBASE-14895
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>
>         Attachments: HBASE-14895.patch, HBASE-14895_1.patch, HBASE-14895_1.patch, HBASE-14895_2.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)