hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashu Pachauri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14953) HBaseInterClusterReplicationEndpoint: Do not retry the whole batch of edits in case of RejectedExecutionException
Date Thu, 10 Dec 2015 19:40:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051522#comment-15051522
] 

Ashu Pachauri commented on HBASE-14953:
---------------------------------------

If we use a bounded queue with fixed bounds, we will encounter the same issue, just a lot
less frequently. But, yeah, your idea of unbounded queue certainly makes sense. We will anyways
have a maximum of  (number of WAL groups)*maxThreads number of items in the queue. I will
make the change.

> HBaseInterClusterReplicationEndpoint: Do not retry the whole batch of edits in case of
RejectedExecutionException
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14953
>                 URL: https://issues.apache.org/jira/browse/HBASE-14953
>             Project: HBase
>          Issue Type: Bug
>          Components: Replication
>    Affects Versions: 2.0.0, 1.2.0, 1.3.0
>            Reporter: Ashu Pachauri
>            Assignee: Ashu Pachauri
>            Priority: Critical
>         Attachments: HBASE-14953-V1.patch
>
>
> When we have wal provider set to multiwal, the ReplicationSource has multiple worker
threads submitting batches to HBaseInterClusterReplicationEndpoint. In such a scenario, it
is quite common to encounter RejectedExecutionException because it takes quite long for shipping
edits to peer cluster compared to reading edits from source and submitting more batches to
the endpoint. 
> The logs are just filled with warnings due to this very exception.
> Since we subdivide batches before actually shipping them, we don't need to fail and resend
the whole batch if one of the sub-batches fails with RejectedExecutionException. Rather, we
should just retry the failed sub-batches. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message