hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12636) Avoid too many write operations on zookeeper in replication
Date Thu, 03 Aug 2017 01:42:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112050#comment-16112050
] 

Hadoop QA commented on HBASE-12636:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  0s{color} | {color:blue}
Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue}  0m  4s{color} | {color:blue}
The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  6s{color} | {color:red}
HBASE-12636 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-12636 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12685233/HBASE-12635-v2.diff
|
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/7903/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> Avoid too many write operations on zookeeper in replication
> -----------------------------------------------------------
>
>                 Key: HBASE-12636
>                 URL: https://issues.apache.org/jira/browse/HBASE-12636
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.94.11
>            Reporter: Liu Shaohui
>            Assignee: Liu Shaohui
>              Labels: replication
>         Attachments: HBASE-12635-v2.diff, HBASE-12636-v1.diff
>
>
> In our production cluster, we found there are about over 1k write operations per second
on zookeeper from hbase replication. The reason is that the replication source will write
the log position to zookeeper for every edit shipping. If the current replicating WAL is just
the WAL that regionserver is writing to,  each skipping will be very small but the frequency
is very high, which causes many write operations on zookeeper.
> A simple solution is that writing log position to zookeeper when position diff or skipped
edit number is larger than a threshold, not every  edit shipping.
> Suggestions are welcomed, thx~



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message