hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phil Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15995) Separate replication WAL reading from shipping
Date Fri, 06 Jan 2017 05:41:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803641#comment-15803641
] 

Phil Yang commented on HBASE-15995:
-----------------------------------

Hi

I have a plan to reduce WAL reading threads to reduce disk IO when we have many peers ( will
create a new issue soon). Your work which separate WAL reading and entries pushing helps a
lot. :)

Now we have two threads so ReplicationSource may be need to change a better name? We have
one thread to read logs and one thread to push logs. One is ReplicationWALReader and another
is ReplicationEntriesPusher? What do you think?	

TestSerialReplication failed, please fix it. I can give some help if you need.

> Separate replication WAL reading from shipping
> ----------------------------------------------
>
>                 Key: HBASE-15995
>                 URL: https://issues.apache.org/jira/browse/HBASE-15995
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Replication
>    Affects Versions: 2.0.0
>            Reporter: Vincent Poon
>            Assignee: Vincent Poon
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15995.master.v1.patch, replicationV1_100ms_delay.png, replicationV2_100ms_delay.png
>
>
> Currently ReplicationSource reads edits from the WAL and ships them in the same thread.
> By breaking out the reading from the shipping, we can introduce greater parallelism and
lay the foundation for further refactoring to a pipelined, streaming model.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message