From user-return-14779-archive-asf-public=cust-asf.ponee.io@storm.apache.org Sun Sep 13 19:23:23 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mailroute1-lw-us.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 806D418063D for ; Sun, 13 Sep 2020 21:23:23 +0200 (CEST) Received: from mail.apache.org (localhost [127.0.0.1]) by mailroute1-lw-us.apache.org (ASF Mail Server at mailroute1-lw-us.apache.org) with SMTP id 988D4122077 for ; Sun, 13 Sep 2020 19:23:20 +0000 (UTC) Received: (qmail 29524 invoked by uid 500); 13 Sep 2020 19:23:19 -0000 Mailing-List: contact user-help@storm.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.apache.org Delivered-To: mailing list user@storm.apache.org Received: (qmail 29514 invoked by uid 99); 13 Sep 2020 19:23:19 -0000 Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Sep 2020 19:23:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id 894931FF42E for ; Sun, 13 Sep 2020 19:23:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org X-Spam-Flag: NO X-Spam-Score: 0.009 X-Spam-Level: X-Spam-Status: No, score=0.009 tagged_above=-999 required=6.31 tests=[KAM_DMARC_STATUS=0.01, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-ec2-va.apache.org ([116.203.227.195]) by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024) with ESMTP id 1B_09yswGwOy for ; Sun, 13 Sep 2020 19:23:18 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=35.164.127.225; helo=omta002.uswest2.a.cloudfilter.net; envelope-from=tomredman@mchsi.com; receiver= Received: from omta002.uswest2.a.cloudfilter.net (omta002.uswest2.a.cloudfilter.net [35.164.127.225]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 8C10FC152F for ; Sun, 13 Sep 2020 19:23:17 +0000 (UTC) Received: from mcc-obgw-5001a.ext.cloudfilter.net ([10.243.65.71]) by cmsmtp with ESMTP id HWLDkvRM21H9YHXahkHnfy; Sun, 13 Sep 2020 19:23:11 +0000 Received: from [192.168.1.4] ([97.64.229.10]) by cmsmtp with ESMTPA id HXadkUzFEToAAHXagkJlqY; Sun, 13 Sep 2020 19:23:11 +0000 X-MCC-ORCPT: user@storm.apache.org X-Authority-Analysis: v=2.4 cv=MtzsV0We c=1 sm=1 tr=0 ts=5f5e719f cx=a_idp_d a=xvPAFWtIcx9H+I5lGvK/BA==:117 a=xvPAFWtIcx9H+I5lGvK/BA==:17 a=IkcTkHD0fZMA:10 a=l5Cd0hxxMeuTBEpiQb8A:9 a=QEXdDO2ut3YA:10 From: "Thomas L. Redman" Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Nodes underutilized Message-Id: <83555851-7287-42B2-8A06-A1A3CDB498B2@mchsi.com> Date: Sun, 13 Sep 2020 14:23:07 -0500 To: user@storm.apache.org X-Mailer: Apple Mail (2.3608.120.23.2.1) X-CMAE-Envelope: MS4xfNp3dYmwfrFgiLcqlrQpqY0S7Iw7YrYsjmNr8yGJzCemJs8JsKIn1kgtapKmtZZlic7VaFvb3a7WHsMbADL4W8/ikmxQR4gxmIG+0gV73SdrGoFrAIgS 5ULXp5qOZDDc8izClu0kTARYeKF26vMEQAjYEmNpcCcjCABTTpjS5R9bjONLvsWg/+e0Yf66cig2pg== Sorry, I had previously sent this from a different email address, not = sure how well that would work with this service, hence this re-send. I=E2=80=99m running storm on a 3 node cluster, 32 physical cores in each = node. I have a complex topology with one spout which is a singleton, = connected to several other bolts most all of which doing natural = language processing. Most of these are pretty heavy weight. The input = spout is easily capable of outpacing the downstream bolts. I get good = performance, but on only one node, even though I specify 3 worker nodes = for my topology. StormUI indicates for any given component that the = executors for that token on the idle machines have emitted very few = tokens, and have transferred none! When I look at the machine usage with htop, I see indeed only one of the = nodes is really getting any usage at all. My heaviest computation nodes = have a very high capacity value. But the machine which hosts the spout = is pegged with significant load. I have used almost exclusively = shuffle(I prefer localOrShuffleGrouping) grouping, but that doesn=E2=80=99= t help. I will have machines that are simply receiving few tuples to = operate on, and those few tuples are not transferred (and I admit I = don=E2=80=99t know quite what that means). So, I have two questions: 1) Why would a component on a node remote from the spout have a lower = Emitted count, and have a Transferred count always at zero? 2) What might cause my high capacity (typically over 1) to not be = offloaded to a more idle machine?=