hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashu Pachauri (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-14953) HBaseInterClusterReplicationEndpoint: Do not retry the whole batch of edits in case of RejectedExecutionException
Date Tue, 08 Dec 2015 23:04:11 GMT

     [ https://issues.apache.org/jira/browse/HBASE-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ashu Pachauri updated HBASE-14953:
----------------------------------
    Attachment: HBASE-14953-V1.patch

V1: Retries remaining batches after sleeping in case we fail submitting new batches. 
One line of thought is we could try using another custom policy of threadpoolexecutor, that
blocks on RejectedExecutionException. However, I don't feel comfortable with that, because
such an implementation is prone to side-effects.

> HBaseInterClusterReplicationEndpoint: Do not retry the whole batch of edits in case of
RejectedExecutionException
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14953
>                 URL: https://issues.apache.org/jira/browse/HBASE-14953
>             Project: HBase
>          Issue Type: Bug
>          Components: Replication
>    Affects Versions: 2.0.0, 1.2.0, 1.3.0
>            Reporter: Ashu Pachauri
>            Assignee: Ashu Pachauri
>         Attachments: HBASE-14953-V1.patch
>
>
> When we have wal provider set to multiwal, the ReplicationSource has multiple worker
threads submitting batches to HBaseInterClusterReplicationEndpoint. In such a scenario, it
is quite common to encounter RejectedExecutionException because it takes quite long for shipping
edits to peer cluster compared to reading edits from source and submitting more batches to
the endpoint. 
> The logs are just filled with warnings due to this very exception.
> Since we subdivide batches before actually shipping them, we don't need to fail and resend
the whole batch if one of the sub-batches fails with RejectedExecutionException. Rather, we
should just retry the failed sub-batches. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message