zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ZOOKEEPER-2251) Add Client side packet response timeout to avoid infinite wait.
Date Fri, 09 Dec 2016 06:58:59 GMT

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15734535#comment-15734535

ASF GitHub Bot commented on ZOOKEEPER-2251:

Github user hanm commented on a diff in the pull request:

    --- Diff: src/java/main/org/apache/zookeeper/ClientCnxn.java ---
    @@ -1495,10 +1504,21 @@ public ReplyHeader submitRequest(RequestHeader h, Record request,
             Packet packet = queuePacket(h, r, request, response, null, null, null,
                     null, watchRegistration, watchDeregistration);
             synchronized (packet) {
    +            long waitStartTime = System.currentTimeMillis();
                 while (!packet.finished) {
    -                packet.wait();
    +                packet.wait(requestTimeout);
    --- End diff --
    This unconditionally adds a timeout is a pretty significant semantic change. Whether or
not we should wait indefinitely is a separate issue to discuss later, but from a compatibility
point of view we'd at least provide a configuration option to let user opt in this feature
rather than having this turned on by default. So by default there is no timeout and client
still waits indefinitely, user who wants to opt in to turn on the timeout needs to explicit
say so and provide a value they think make sense to them. Otherwise, we'd have to solve the
problem of what the default timeout value we should set in ZK config if this feature is turned
on by default, and that problem is hard and very likely there is no default value that makes
everyone happy.

> Add Client side packet response timeout to avoid infinite wait.
> ---------------------------------------------------------------
>                 Key: ZOOKEEPER-2251
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2251
>             Project: ZooKeeper
>          Issue Type: Bug
>          Components: java client
>    Affects Versions: 3.4.9, 3.5.2
>            Reporter: nijel
>            Assignee: Arshad Mohammad
>            Priority: Critical
>              Labels: fault
>             Fix For: 3.4.10, 3.5.3, 3.6.0
>         Attachments: ZOOKEEPER-2251-01.patch, ZOOKEEPER-2251-02.patch, ZOOKEEPER-2251-03.patch,
> I came across one issue related to Client side packet response timeout In my cluster
many packet drops happened for some time.
> One observation is the zookeeper client got hanged. As per the thread dump it is waiting
for the response/ACK for the operation performed (synchronous API used here).
> I am using zookeeper.serverCnxnFactory=org.apache.zookeeper.server.NIOServerCnxnFactory
> Since only few packets missed there is no DISCONNECTED event occurred.
> Need add a "response time out" for the operations or packets.
> *Comments from [~rakeshr]*
> My observation about the problem:-
> * Can use tools like 'Wireshark' to simulate the artificial packet loss.
> * Assume there is only one packet in the 'outgoingQueue' and unfortunately the server
response packet lost. Now, client will enter into infinite waiting. https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1515
> * Probably we can discuss more about this problem and possible solutions(add packet ACK
timeout or another better approach) in the jira.

This message was sent by Atlassian JIRA

View raw message