From notifications-return-1365-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Tue Aug 13 22:24:07 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 3E0B51804BB for ; Wed, 14 Aug 2019 00:24:07 +0200 (CEST) Received: (qmail 28430 invoked by uid 500); 13 Aug 2019 22:24:06 -0000 Mailing-List: contact notifications-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zookeeper.apache.org Delivered-To: mailing list notifications@zookeeper.apache.org Received: (qmail 28420 invoked by uid 99); 13 Aug 2019 22:24:06 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Aug 2019 22:24:06 +0000 From: GitBox To: notifications@zookeeper.apache.org Subject: [GitHub] [zookeeper] jhuan31 commented on a change in pull request #1051: ZOOKEEPER-3503: Add server side large request throttling Message-ID: <156573504643.13839.7200162657348559053.gitbox@gitbox.apache.org> Date: Tue, 13 Aug 2019 22:24:06 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit jhuan31 commented on a change in pull request #1051: ZOOKEEPER-3503: Add server side large request throttling URL: https://github.com/apache/zookeeper/pull/1051#discussion_r313637988 ########## File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java ########## @@ -221,6 +221,37 @@ private RequestThrottler requestThrottler; public static final String SNAP_COUNT = "zookeeper.snapCount"; + /** + * This setting sets a limit on the total number of large requests that + * can be inflight and is designed to prevent ZooKeeper from accepting + * too many large requests such that the JVM runs out of usable heap and + * ultimately crashes. + * + * The limit is enforced by the {@link checkRequestSize(int, boolean)} + * method which is called by the connection layer ({@link NIOServerCnxn}, + * {@link NettyServerCnxn}) before allocating a byte buffer and pulling + * data off the TCP socket. The limit is then checked again by the + * ZooKeeper server in {@link processPacket(ServerCnxn, ByteBuffer)} which + * also atomically updates {@link currentLargeRequestBytes}. The request is + * then marked as a large request, with the request size stored in the Request + * object so that it can later be decremented from {@link currentLargeRequestsBytes}. + * + * When a request is completed or dropped, the relevant code path calls the + * {@link requestFinished(Request)} method which performs the decrement if + * needed. + */ + private volatile int largeRequestMaxBytes = + Integer.getInteger("zookeeper.largeRequestMaxBytes", 100 * 1024 * 1024); + + /** + * The size threshold after which a request is considered a large request + * and is checked against the large request byte limit. + */ + private volatile int largeRequestThreshold = Review comment: I add some basic checking and printing the current settings to the log. It's really hard to say what the valid ranges should be. Also I feel throwing an exception is a little too harsh. I just disable the large request throttling if the values are wrong. What do you think? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services