hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17314) Limit total buffered size for all replication sources
Date Wed, 21 Dec 2016 16:33:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15767488#comment-15767488
] 

stack commented on HBASE-17314:
-------------------------------

[~yangzhe1991] If making an addendum for hanging test, here is some other input.

HConstants is for defines that  are used in many places. The preference is to keep defines
beside the code where they are used so move these to ReplicationSource

935	  public static final String REPLICATION_SOURCE_TOTAL_BUFFER_KEY = "replication.total.buffer.quota";
936	  public static final int REPLICATION_SOURCE_TOTAL_BUFFER_DFAULT = 256 * 1024 * 1024;

Suggest too that the explanation for why 256M that you give above be written as a comment
on the above define.

This looks like it could be package private rather than public:

2344	  public ReplicationSourceService getReplicationSourceService() {

Otherwise patch looks good.  Pity that replication is so hard to test. Any ideas on how to
make it easier? Test could be hanging for any of many reasons given you have to put up two
clusters inside one jvm.

> Limit total buffered size for all replication sources
> -----------------------------------------------------
>
>                 Key: HBASE-17314
>                 URL: https://issues.apache.org/jira/browse/HBASE-17314
>             Project: HBase
>          Issue Type: Improvement
>          Components: Replication
>            Reporter: Phil Yang
>            Assignee: Phil Yang
>             Fix For: 2.0.0, 1.4.0
>
>         Attachments: HBASE-17314.branch-1.v01.patch, HBASE-17314.v01.patch, HBASE-17314.v02.patch,
HBASE-17314.v03.patch, HBASE-17314.v04.patch
>
>
> If we have many peers or some servers have many recovered queues, we will hold many entries
in memory which will increase the pressure of GC, even maybe OOM because we will read entries
for 64MB to buffer in default for one source.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message