Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@phoenix.apache.org
Date: Fri, 7 Jul 2017 18:13:00 +0000 (UTC)
From: "James Taylor (JIRA)" <jira@apache.org>
To: dev@phoenix.apache.org
Message-ID: <JIRA.13085296.1499366185000.195829.1499451180137@Atlassian.JIRA>
In-Reply-To: <JIRA.13085296.1499366185000@Atlassian.JIRA>
References: <JIRA.13085296.1499366185000@Atlassian.JIRA> <JIRA.13085296.1499366185107@jira-lw-us.apache.org>
Subject: [jira] [Commented] (PHOENIX-3997)
 UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check
 the memstore size and wait for flush.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 07 Jul 2017 18:13:04 -0000


    [ https://issues.apache.org/jira/browse/PHOENIX-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078477#comment-16078477 ] 

James Taylor commented on PHOENIX-3997:
---------------------------------------

Just to confirm, [~sergey.soldatov] - this throttling we're doing is for cases in which we're reading and writing to the same region server, right? Even for the local index case, we'd be writing to a different region in a different store, but we know it'll be on the same region server.

The commitBatchWithHTable is for cases in which we don't know whether the region we're writing to is on the same or a different region server. Worse-case, if we're writing to a new table, all writes could potentially be going to the same region server from which we're reading (at least until splits occur that move the region to a different region server).

If the throttling is for reads/writes to the same region server, then I think we need more testing around the above case before we remove it.

> UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check the memstore size and wait for flush. 
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3997
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3997
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Thomas D'Silva
>            Assignee: Geoffrey Jacoby
>             Fix For: 4.12.0
>
>         Attachments: PHOENIX-3997.patch
>
>
> [~ankit.singhal] 
> In UngroupedAggregateRegionObserver.commitBatchWithHTable() do we need to check the memstore size and wait for the flush. We are using a htable to write the mutations.
> {code}
>  // When memstore size reaches blockingMemstoreSize we are waiting 3 seconds for the
>         // flush happen which decrease the memstore size and then writes allowed on the region.
>         for (int i = 0; region.getMemstoreSize().get() > blockingMemstoreSize && i < 30; i++) {
>             try {
>                 checkForRegionClosing();
>                 Thread.sleep(100);
>             } catch (InterruptedException e) {
>                 Thread.currentThread().interrupt();
>                 throw new IOException(e);
>             }
>         }
>         logger.debug("Committing batch of " + mutations.size() + " mutations for " + table);
>         try {
>             table.batch(mutations);
>         } catch (InterruptedException e) {
>             throw new RuntimeException(e);
>         }
> {code}
> FYI [~jamestaylor] [~apurtell]


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)