zookeeper-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [zookeeper] jhuan31 commented on a change in pull request #1051: ZOOKEEPER-3503: Add server side large request throttling
Date Tue, 13 Aug 2019 22:24:06 GMT
jhuan31 commented on a change in pull request #1051: ZOOKEEPER-3503: Add server side large
request throttling
URL: https://github.com/apache/zookeeper/pull/1051#discussion_r313637988
 
 

 ##########
 File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
 ##########
 @@ -221,6 +221,37 @@
     private RequestThrottler requestThrottler;
     public static final String SNAP_COUNT = "zookeeper.snapCount";
 
+    /**
+     * This setting sets a limit on the total number of large requests that
+     * can be inflight and is designed to prevent ZooKeeper from accepting
+     * too many large requests such that the JVM runs out of usable heap and
+     * ultimately crashes.
+     *
+     * The limit is enforced by the {@link checkRequestSize(int, boolean)}
+     * method which is called by the connection layer ({@link NIOServerCnxn},
+     * {@link NettyServerCnxn}) before allocating a byte buffer and pulling
+     * data off the TCP socket. The limit is then checked again by the
+     * ZooKeeper server in {@link processPacket(ServerCnxn, ByteBuffer)} which
+     * also atomically updates {@link currentLargeRequestBytes}. The request is
+     * then marked as a large request, with the request size stored in the Request
+     * object so that it can later be decremented from {@link currentLargeRequestsBytes}.
+     *
+     * When a request is completed or dropped, the relevant code path calls the
+     * {@link requestFinished(Request)} method which performs the decrement if
+     * needed.
+     */
+    private volatile int largeRequestMaxBytes =
+        Integer.getInteger("zookeeper.largeRequestMaxBytes", 100 * 1024 * 1024);
+
+    /**
+     * The size threshold after which a request is considered a large request
+     * and is checked against the large request byte limit.
+     */
+    private volatile int largeRequestThreshold =
 
 Review comment:
   I add some basic checking and printing the current settings to the log. It's really hard
to say what the valid ranges should be. Also I feel throwing an exception is a little too
harsh. I just disable the large request throttling if the values are wrong. What do you think?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message