phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3997) UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check the memstore size and wait for flush.
Date Fri, 07 Jul 2017 18:40:01 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078504#comment-16078504
] 

James Taylor commented on PHOENIX-3997:
---------------------------------------

An UPSERT SELECT to the same table which modifies row key columns calls commitBatchWithHTable,
so we can be writing to the same table. We could potentially refactor the call to commitBatchWithHTable
from commit to not call it.

If we're writing to a separate table, is there any need for this throttling in the case that
the source and target tables happen to be reading and writing regions hosted on the same region
server? 

> UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check the memstore
size and wait for flush. 
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3997
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3997
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Thomas D'Silva
>            Assignee: Geoffrey Jacoby
>             Fix For: 4.12.0
>
>         Attachments: PHOENIX-3997.patch
>
>
> [~ankit.singhal] 
> In UngroupedAggregateRegionObserver.commitBatchWithHTable() do we need to check the memstore
size and wait for the flush. We are using a htable to write the mutations.
> {code}
>  // When memstore size reaches blockingMemstoreSize we are waiting 3 seconds for the
>         // flush happen which decrease the memstore size and then writes allowed on the
region.
>         for (int i = 0; region.getMemstoreSize().get() > blockingMemstoreSize &&
i < 30; i++) {
>             try {
>                 checkForRegionClosing();
>                 Thread.sleep(100);
>             } catch (InterruptedException e) {
>                 Thread.currentThread().interrupt();
>                 throw new IOException(e);
>             }
>         }
>         logger.debug("Committing batch of " + mutations.size() + " mutations for " +
table);
>         try {
>             table.batch(mutations);
>         } catch (InterruptedException e) {
>             throw new RuntimeException(e);
>         }
> {code}
> FYI [~jamestaylor] [~apurtell]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message