phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3997) UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check the memstore size and wait for flush.
Date Thu, 06 Jul 2017 22:18:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077271#comment-16077271
] 

James Taylor commented on PHOENIX-3997:
---------------------------------------

Ah, yes - thanks for the reminder, [~sergey.soldatov]. See also this comment on the lock member
variable:
{code}
    /**
     * This lock used for synchronizing the state of
     * {@link UngroupedAggregateRegionObserver#scansReferenceCount},
     * {@link UngroupedAggregateRegionObserver#isRegionClosing} variables used to avoid possible
     * dead lock situation in case below steps: 
     * 1. We get read lock when we start writing local indexes, deletes etc.. 
     * 2. when memstore reach threshold, flushes happen. Since they use read (shared) lock
they 
     * happen without any problem until someone tries to obtain write lock. 
     * 3. at one moment we decide to split/bulkload/close and try to acquire write lock. 
     * 4. Since that moment all attempts to get read lock will be blocked. I.e. no more 
     * flushes will happen. But we continue to fill memstore with local index batches and

     * finally we get RTBE.
     * 
     * The solution to this is to not allow or delay operations acquire the write lock.
     * 1) In case of split we just throw IOException so split won't happen but it will not
cause any harm.
     * 2) In case of bulkload failing it by throwing the exception. 
     * 3) In case of region close by balancer/move wait before closing the reason and fail
the query which 
     * does write after reading. 
     * 
     * See PHOENIX-3111 for more info.
     */
{code}


> UngroupedAggregateRegionObserver.commitBatchWithHTable() should not check the memstore
size and wait for flush. 
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3997
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3997
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Thomas D'Silva
>            Assignee: Geoffrey Jacoby
>             Fix For: 4.12.0
>
>         Attachments: PHOENIX-3997.patch
>
>
> [~ankit.singhal] 
> In UngroupedAggregateRegionObserver.commitBatchWithHTable() do we need to check the memstore
size and wait for the flush. We are using a htable to write the mutations.
> {code}
>  // When memstore size reaches blockingMemstoreSize we are waiting 3 seconds for the
>         // flush happen which decrease the memstore size and then writes allowed on the
region.
>         for (int i = 0; region.getMemstoreSize().get() > blockingMemstoreSize &&
i < 30; i++) {
>             try {
>                 checkForRegionClosing();
>                 Thread.sleep(100);
>             } catch (InterruptedException e) {
>                 Thread.currentThread().interrupt();
>                 throw new IOException(e);
>             }
>         }
>         logger.debug("Committing batch of " + mutations.size() + " mutations for " +
table);
>         try {
>             table.batch(mutations);
>         } catch (InterruptedException e) {
>             throw new RuntimeException(e);
>         }
> {code}
> FYI [~jamestaylor] [~apurtell]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message