hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liu Shaohui (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-12636) Avoid too many write operations on zookeeper in replication
Date Thu, 04 Dec 2014 12:52:12 GMT
Liu Shaohui created HBASE-12636:
-----------------------------------

             Summary: Avoid too many write operations on zookeeper in replication
                 Key: HBASE-12636
                 URL: https://issues.apache.org/jira/browse/HBASE-12636
             Project: HBase
          Issue Type: Improvement
    Affects Versions: 0.94.11
            Reporter: Liu Shaohui
            Assignee: Liu Shaohui
             Fix For: 1.0.0


In our production cluster, we found there are about over 1k write operations per second on
zookeeper from hbase replication. The reason is that the replication source will write the
log position to zookeeper for every edit shipping. If the current replicating WAL is just
the WAL that regionserver is writing to,  each skipping will be very small but the frequency
is very high, which causes many write operations on zookeeper.

A simple solution is that writing log position to zookeeper when position diff or skipped
edit number is larger than a threshold, not every  edit shipping.

Suggestions are welcomed, thx~





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message