hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vincent Poon (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-15995) Separate replication WAL reading from shipping
Date Thu, 05 Jan 2017 23:11:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802818#comment-15802818
] 

Vincent Poon edited comment on HBASE-15995 at 1/5/17 11:11 PM:
---------------------------------------------------------------

Example of network usage.  X-axis is time and Y-axis is network usage.

The patch keeps the network utilization higher while replicating.  V1 has wider gaps while
reading/filtering the next batch.

V1:
!replicationV1_100ms_delay.png!

V2:
!replicationV2_100ms_delay.png!


was (Author: vincentpoon):
Example of network usage.  X-axis is time and Y-axis is network usage.

The patch keeps the network utilization higher while replicating.  V1 has wider gaps while
reading/filtering the next batch.

> Separate replication WAL reading from shipping
> ----------------------------------------------
>
>                 Key: HBASE-15995
>                 URL: https://issues.apache.org/jira/browse/HBASE-15995
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Replication
>    Affects Versions: 2.0.0
>            Reporter: Vincent Poon
>            Assignee: Vincent Poon
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15995.master.v1.patch, replicationV1_100ms_delay.png, replicationV2_100ms_delay.png
>
>
> Currently ReplicationSource reads edits from the WAL and ships them in the same thread.
> By breaking out the reading from the shipping, we can introduce greater parallelism and
lay the foundation for further refactoring to a pipelined, streaming model.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message