From user-return-12180-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Fri Sep 27 19:07:28 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 2ACF6180638 for ; Fri, 27 Sep 2019 21:07:28 +0200 (CEST) Received: (qmail 45268 invoked by uid 500); 27 Sep 2019 19:07:27 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 45253 invoked by uid 99); 27 Sep 2019 19:07:26 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Sep 2019 19:07:26 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 3B5C8C073C for ; Fri, 27 Sep 2019 19:07:26 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.05 X-Spam-Level: ** X-Spam-Status: No, score=2.05 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id Smyb55Q8aV8g for ; Fri, 27 Sep 2019 19:07:23 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::434; helo=mail-wr1-x434.google.com; envelope-from=shyue2010@gmail.com; receiver= Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 4231B7DDB1 for ; Fri, 27 Sep 2019 19:07:23 +0000 (UTC) Received: by mail-wr1-x434.google.com with SMTP id i18so4349506wru.11 for ; Fri, 27 Sep 2019 12:07:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=SIlDKdP0/XkWPyqjrmqxBaSPAxPq/2rcRUfgNHsuXxM=; b=XKi51a1zWFsDMXJ+cqHK+ySLTh8xLLLnA4AVV+g5qq7tJw6XJinZCjfYEn6bRJfksL GI30G00W0GNK8Ttq54ipG2X+bg1kRhscgza7FC6fkfltEALQFUfsKTxTZEL6yCdJA2q8 uo7A2GCLEGp+sFmlulb2JEO3BGvZciSCqHdBT15ZxcUV2raqrMqpy2bZLZpnt7EiDOn2 5Hk7Uy1a0VNPpi6kx5UuKZhg/CcRqf8Gt2KIPOGJM2CB+1+/Nwc8zKExnqSBdRz45ab3 RbuVW7h1SoABq30aRp7DYWgzAMsux5FrpFKvGRw28fUCWaBww+FC/7eXYJTlg/mieRMF MzmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=SIlDKdP0/XkWPyqjrmqxBaSPAxPq/2rcRUfgNHsuXxM=; b=RP1gXjWIQXON1ZVKOtrga5Py4p9KO2u5p4m6QiBIBNKZ3rzKYizolKrwQp0tOpF8Yx gwzwPEchgmLf0Nydi4+fhsXmK8/KYUO4DH0InonZUIhaodYPO8RmXGtIeDiiPqrGoO8g ahW7AgZdj2ViPrP2CY5Xng3e+HjNHVdpL3nEyyylW+siA2RKNY/ZBmeBfR8Le4Br/R25 /pqbBUKbN6bNSz2SkBHibBNE8ZNnbNQbLlBhYiC66Cn1I2Du8Kdm0cZ7YbLpt600K9p/ vJnrOMmFIiBUWKucSIgNxfLiumJdvZMTfi28D4tEKk/tb8YAy1/+sc2KEAOJ9ywS3wCR 4nPQ== X-Gm-Message-State: APjAAAWLVIbbPYX7p5uxdh/HTjYg3P/sleIt2/3sKafP3JbGEKj6IhzR MM28lOLOooFOHhNR2a8GrbZxH+kl7QelXJK1+WFdYpMoN6w= X-Google-Smtp-Source: APXvYqxjgypz10AlYKg7prn6ePP0PKWjYNoWdmE2ixaByK8jqKByXlPfmxSzQIsRRRmsLFCtB4g+3HnGI4rbY15MTGM= X-Received: by 2002:adf:d84f:: with SMTP id k15mr4024847wrl.70.1569611242445; Fri, 27 Sep 2019 12:07:22 -0700 (PDT) MIME-Version: 1.0 References: <6E602A7A-1AA7-417C-8B4E-F41A7B6D9243@gmail.com> In-Reply-To: <6E602A7A-1AA7-417C-8B4E-F41A7B6D9243@gmail.com> From: Yue Shen Date: Fri, 27 Sep 2019 12:07:11 -0700 Message-ID: Subject: Re: How to scale ZooKeeper to support 10K concurrent connections? To: user@zookeeper.apache.org Content-Type: multipart/alternative; boundary="0000000000008aad9805938d9831" --0000000000008aad9805938d9831 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thank you, Jorn. We don't use Solr. We inherited this architecture from another team, and we don't have time to redesign a new system to scale in 2 months. As you said, if I were to design it, I would definitely put a queue in front of Lambda service, our new design is actually on the way with Kafka upfront. However we need to scale it out with the coming holiday season before we can roll out the new system, which is just kicked off a couple of weeks ago. At this point, we want to tune ZooKeeper so it can handle 10K concurrent calls. Any suggestions? Thank you, Yue On Fri, Sep 27, 2019 at 10:39 AM J=C3=B6rn Franke wr= ote: > Put the Solr request on a SQS queue using your 10k instances and have 10 > or so worker working on the queue to put it in Solr. Having 10k connectio= ns > just because lambda creates that many instances does not make sense for n= o > database service. > > > Am 27.09.2019 um 19:01 schrieb Yue Shen : > > > > =EF=BB=BFDear ZooKeeper users, > > > > I have a special use case, in which I use AWS lambda service. > > > > Inside the lambda service logic, it goes to ZooKeeper to check the work= er > > for the data, if exists, connect to the worker endpoint and send the > data. > > If the worker isn't assigned, the logic will post a new assignment, and > > wait for it to be assigned to a worker. There is a coordinator to watch > the > > new assignment and assign tasks. > > > > My problem comes with AWS Lambda service, which can launch tens of > > thousands of calls. When this happens, I found many calls get timeout. > The > > active connections to ZooKeeper plateau around 6500. > > > > BTW, I run ZooKeeper as 3 node ensemble, run on Quorum. > > > > How can I scale ZooKeeper to support more concurrent connections? > > > > Thank you, > > Yue > --0000000000008aad9805938d9831--