hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phil Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15995) Separate replication WAL reading from shipping
Date Sun, 08 Jan 2017 03:17:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15808590#comment-15808590

Phil Yang commented on HBASE-15995:

I think we should find a way to make global entries size limiter still working in the new
model. The queue size is configurable so the size may be larger than double, and even if we
have HBASE-17432 there are still failover peers so there is still risk for OOM. Maybe we can
check AtomitLong each time when we want to read next entry. Now we only check it after reading
an entry and push if exeed. Double check may be mush safer?

> Separate replication WAL reading from shipping
> ----------------------------------------------
>                 Key: HBASE-15995
>                 URL: https://issues.apache.org/jira/browse/HBASE-15995
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Replication
>    Affects Versions: 2.0.0
>            Reporter: Vincent Poon
>            Assignee: Vincent Poon
>             Fix For: 2.0.0
>         Attachments: HBASE-15995.master.v1.patch, HBASE-15995.master.v2.patch, replicationV1_100ms_delay.png,
> Currently ReplicationSource reads edits from the WAL and ships them in the same thread.
> By breaking out the reading from the shipping, we can introduce greater parallelism and
lay the foundation for further refactoring to a pipelined, streaming model.

This message was sent by Atlassian JIRA

View raw message