zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Shraer (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (ZOOKEEPER-1609) Improve ZooKeeper performance under mixed workload
Date Fri, 07 Apr 2017 22:51:41 GMT

     [ https://issues.apache.org/jira/browse/ZOOKEEPER-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Shraer resolved ZOOKEEPER-1609.
-----------------------------------------
    Resolution: Duplicate

I think that this is very similar to what was done with ZOOKEEPER-2024. 

It is still the case that the FinalRequestProcessor only accepts a single write request or
multiple reads, and this could perhaps be optimized (though not the topic of this JIRA), but
this would be complex and the benefit is unclear. 

> Improve ZooKeeper performance under mixed workload
> --------------------------------------------------
>
>                 Key: ZOOKEEPER-1609
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1609
>             Project: ZooKeeper
>          Issue Type: Improvement
>          Components: server
>    Affects Versions: 3.4.3
>            Reporter: Thawan Kooburat
>
> ZOOKEEPER-1505 allows 1 write or N reads to pass through the CommitProcessor at any given
time. I did performance experiment similar to http://wiki.apache.org/hadoop/ZooKeeper/Performance
and found that read throughput drop dramatically when there are write requests. After a bit
more investigation, I found that
> the biggest bottleneck is at the request queue entering the CommitProcessor.
> When the CommitProcessor see any write request, it will need to block the entire pipeline
and wait until matching commit from the leader. This means that all read requests (including
ping request) won't be able to go through. The time spent waiting for commit from the leader
far exceed the time spent waiting for 1 write to goes through the CommitProcessor. 
> The current plan is to create multiple request queues at the front of the CommitProcessor.
Requests are hashed using sessionId and send to one of the queue. Whenever, the CommitProcessor
saw a write request on one of the queue it moves on to process read requests. It will have
to unblock the write requests in the same order that it sent to the leader, so it may need
to maintain a separate list to keep track of that. 
> The correctness is the same as having more learners in the ensemble. Sessions which are
hashed onto a different queue is similar to sessions connecting to a different learners in
the ensemble. 
> I am hoping that this will improve read throughput and reduce disconnect rate on an ensemble
with large number of clients  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message