Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id ED0D317D0A for ; Wed, 15 Apr 2015 20:17:12 +0000 (UTC) Received: (qmail 1663 invoked by uid 500); 15 Apr 2015 20:17:12 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 1609 invoked by uid 500); 15 Apr 2015 20:17:12 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 1598 invoked by uid 99); 15 Apr 2015 20:17:12 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Apr 2015 20:17:12 +0000 X-ASF-Spam-Status: No, hits=3.7 required=5.0 tests=FORGED_YAHOO_RCVD,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_REPLYTO_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ppine7@yahoo.com designates 216.109.114.57 as permitted sender) Received: from [216.109.114.57] (HELO nm41.bullet.mail.bf1.yahoo.com) (216.109.114.57) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Apr 2015 20:17:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1429128941; bh=tuK0wW3NTY0r8Pdujrz1RjG0pbOSpgr9VkI10Nav+j4=; h=Date:From:Reply-To:To:Subject:From:Subject; b=msAgclJz4au0em0d5SHaT0ft3YgR5nNXgY4p4JaeiJeZO/OUmRcEqMbp1dQMO4Pbhn5SQEte/PX+fuVPJ1RCJCxqK/qmjKYD3+jzho7xKT5yneMg3ew3syGVG01QwRVUGx9pD5EzBc8Vb8fmYxrlcn6pAtQTE98twiwU6qP02oCcxpAmztfI4IRDB+MnGbWNg/7b3LASV4ZyNKX59lK5Evo6tdL1hEAlUVaxSZF973g4FNCIaBFDaW+8U5F6v55KHtidHCSKxPVgZcQ6KDo7VdEgzW62bIE+/vk21s3fZpd8VAXSXniOxkmh1fNV9AnV+H7UrFMiI3fdf3bqXPCVRA== Received: from [98.139.214.32] by nm41.bullet.mail.bf1.yahoo.com with NNFMP; 15 Apr 2015 20:15:41 -0000 Received: from [98.139.212.228] by tm15.bullet.mail.bf1.yahoo.com with NNFMP; 15 Apr 2015 20:15:40 -0000 Received: from [127.0.0.1] by omp1037.mail.bf1.yahoo.com with NNFMP; 15 Apr 2015 20:15:40 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 977162.82512.bm@omp1037.mail.bf1.yahoo.com X-YMail-OSG: YZQg6wcVM1l50KqIyBI5SoHcrb6YEjEpXlo7sRmAoszOLqo5IW1dTJ90M7dOpGl P04KUpGJtGg42_HAf0UjB0qNwYB3UapajQTcQX_rX65_yxXhajo4l0YUgX1HFN1S0aIkwYxeZWJa bq5b4QZOS4fvHec0l6y7rVesIbOi_IwT2_k.sY8sMBp3NxjP8wfszte2j3OSxy6RNXCujoyo9ARF e1342SecifS_kR0pp2850bVrMedudFYqkOf19sdzUVYCCWKmQrNRJsOoZ6Z1ywwpiOU77nUnwS3C 4PulKeuIDtYmtIUfQPpJESOcSj5uZf2YQuNKUlvIiV8Q5OVPwVC1iiZDHPFW2H8rnx1VDG3R8Uha XVcUjRGb_HPbSnL1p7aD.PtoGFsS8h8lZGIYWO7dNHeDlroEBvsx7v24hA8t5K6A3ZqN1i2ALa3N 2xgp1aVmHdV3qhq3ZJHHeFrhcPORp3dxHlXw63_x6WX4QHNWkjP1JkKe2GKHHXtZjU.dOBTHNn._ s0y9q Received: by 76.13.26.159; Wed, 15 Apr 2015 20:15:40 +0000 Date: Wed, 15 Apr 2015 20:15:40 +0000 (UTC) From: Marina Reply-To: Marina To: User Message-ID: <568342469.4648108.1429128940218.JavaMail.yahoo@mail.yahoo.com> Subject: ElasticSearch workers thread number? MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_4648107_556550597.1429128940216" X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_4648107_556550597.1429128940216 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable One more question on Flume ElasticSearchSink. I see that for HDFS sink, you= can control the number of worker threads using this parameter: hdfs.thread= sPoolSize. However, I do not see any such parameter for ElasticSearchSink..= .. Does it mean that ElasticSsearchSink will span only one thread to push e= vents form Flume to ES? If I have rates of a few hundred events per second = (say, up to 3K) - this will become a performance bottleneck. thanks!Marina =C2=A0 ------=_Part_4648107_556550597.1429128940216 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit
One more question on Flume ElasticSearchSink. I see that for HDFS sink, you can control the number of worker threads using this parameter: hdfs.threadsPoolSize. However, I do not see any such parameter for ElasticSearchSink.... Does it mean that ElasticSsearchSink will span only one thread to push events form Flume to ES? If I have rates of a few hundred events per second (say, up to 3K) - this will become a performance bottleneck.

thanks!
Marina
 
------=_Part_4648107_556550597.1429128940216--