Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 19 Apr 2017 04:41:42 +0000 (UTC)
From: "Abhishek Singh Chouhan (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13064690.1492519438000.314417.1492576902333@Atlassian.JIRA>
In-Reply-To: <JIRA.13064690.1492519438000@Atlassian.JIRA>
References: <JIRA.13064690.1492519438000@Atlassian.JIRA> <JIRA.13064690.1492519438099@jira-lw-us.apache.org>
Subject: [jira] [Comment Edited] (HBASE-17937) Memstore size becomes
 negative in case of expensive postPut/Delete Coprocessor call
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 19 Apr 2017 04:41:49 -0000


    [ https://issues.apache.org/jira/browse/HBASE-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974057#comment-15974057 ] 

Abhishek Singh Chouhan edited comment on HBASE-17937 at 4/19/17 4:41 AM:
-------------------------------------------------------------------------

[~lhofhansl] In master the order of operations in mvcc is different from branch-1. We do not apply to memstore first and then if wal sync fails, rollback. Rather we sync to wal first and then write to memstore, hence updating the size just after that step.
[~enis] I'll make the changes and put up a new patch. Initial idea of the test was to have a coprocessor thats slow and at the same time flush should happen, since that was turning a bit difficult thought to trigger flush from the coprocessor itself :D


was (Author: abhishek.chouhan):
[~lhofhansl] In master the order of operations in mvcc is different from branch-1. We do not apply to memstore first and then if wal sync fails, rollback. Rather we sync to wal first and then write to memstore, hence updating the size just after that step.
[~enis] I'll make the changes and put up a new patch. Initial idea of the test was to have a coprocessor thats slow and at the same time flush should happen, since that was turning a bit difficult thought to trigger flush from the coprocess itself :D

> Memstore size becomes negative in case of expensive postPut/Delete Coprocessor call
> -----------------------------------------------------------------------------------
>
>                 Key: HBASE-17937
>                 URL: https://issues.apache.org/jira/browse/HBASE-17937
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 1.3.1, 0.98.24
>            Reporter: Abhishek Singh Chouhan
>            Assignee: Abhishek Singh Chouhan
>         Attachments: HBASE-17937.branch-1.001.patch, HBASE-17937.master.001.patch, HBASE-17937.master.002.patch, HBASE-17937.master.002.patch
>
>
> We ran into a situation where the memstore size became negative due to expensive postPut/Delete Coprocessor calls in doMiniBatchMutate. We update the memstore size in the finally block of doMiniBatchMutate, however a queued flush can be triggered during the coprocessor calls(if they are taking time eg. index updates) since we have released the locks and advanced mvcc at this point. The flush will turn the memstore size negative since the value subtracted is the actual value flushed from stores. The negative value impacts the future flushes amongst others that depend on memstore size.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)