phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3997) UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check the memstore size and wait for flush.
Date Fri, 07 Jul 2017 18:13:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078477#comment-16078477
] 

James Taylor commented on PHOENIX-3997:
---------------------------------------

Just to confirm, [~sergey.soldatov] - this throttling we're doing is for cases in which we're
reading and writing to the same region server, right? Even for the local index case, we'd
be writing to a different region in a different store, but we know it'll be on the same region
server.

The commitBatchWithHTable is for cases in which we don't know whether the region we're writing
to is on the same or a different region server. Worse-case, if we're writing to a new table,
all writes could potentially be going to the same region server from which we're reading (at
least until splits occur that move the region to a different region server).

If the throttling is for reads/writes to the same region server, then I think we need more
testing around the above case before we remove it.

> UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check the memstore
size and wait for flush. 
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3997
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3997
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Thomas D'Silva
>            Assignee: Geoffrey Jacoby
>             Fix For: 4.12.0
>
>         Attachments: PHOENIX-3997.patch
>
>
> [~ankit.singhal] 
> In UngroupedAggregateRegionObserver.commitBatchWithHTable() do we need to check the memstore
size and wait for the flush. We are using a htable to write the mutations.
> {code}
>  // When memstore size reaches blockingMemstoreSize we are waiting 3 seconds for the
>         // flush happen which decrease the memstore size and then writes allowed on the
region.
>         for (int i = 0; region.getMemstoreSize().get() > blockingMemstoreSize &&
i < 30; i++) {
>             try {
>                 checkForRegionClosing();
>                 Thread.sleep(100);
>             } catch (InterruptedException e) {
>                 Thread.currentThread().interrupt();
>                 throw new IOException(e);
>             }
>         }
>         logger.debug("Committing batch of " + mutations.size() + " mutations for " +
table);
>         try {
>             table.batch(mutations);
>         } catch (InterruptedException e) {
>             throw new RuntimeException(e);
>         }
> {code}
> FYI [~jamestaylor] [~apurtell]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message