From user-return-3683-archive-asf-public=cust-asf.ponee.io@giraph.apache.org Wed Dec 12 16:31:24 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 6A62418064E for ; Wed, 12 Dec 2018 16:31:24 +0100 (CET) Received: (qmail 41085 invoked by uid 500); 12 Dec 2018 15:31:23 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 41075 invoked by uid 99); 12 Dec 2018 15:31:23 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Dec 2018 15:31:23 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id EB879180796 for ; Wed, 12 Dec 2018 15:31:22 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.797 X-Spam-Level: * X-Spam-Status: No, score=1.797 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id EGX8IjYJqekm for ; Wed, 12 Dec 2018 15:31:21 +0000 (UTC) Received: from mail-it1-f176.google.com (mail-it1-f176.google.com [209.85.166.176]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 4A4EB5FB08 for ; Wed, 12 Dec 2018 15:31:21 +0000 (UTC) Received: by mail-it1-f176.google.com with SMTP id c9so10418246itj.1 for ; Wed, 12 Dec 2018 07:31:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=6KIaYtYuMwfg4ab6gcZ2Ea6CW865wyNTfAU1mx8vju0=; b=hgxlxJWdSgJtTjr1/X0023maOkwT6dnp5c2XnbzizOTjnq9ctdfNURl/2dyND2eiLj VogrM4i2ku55Fcgx4aB1jGl316M7zRwg8JaNirVTooQum/BzWHeIvbRGTfoL5oiXu3Ti 322mwgqCwhoZGRnP0tZalBDnXix+iZ68g1UQPb5GUIydmlxveye4/MZv5KMfUn03DsqD qtSDeq29aUNJb7SxC8NvbXbIubsN4T3O7d2bIT6sNwF7718J0uQAwWpg+fp+FtnldXeD yRbQ1V2fOfsKQfB5ie80YsZj5yUGAatK3H0ndgrb9A/p16jwdMSaDCMk+w0sCBffl71Q yZfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=6KIaYtYuMwfg4ab6gcZ2Ea6CW865wyNTfAU1mx8vju0=; b=YL6imR2RSjhNlNwpiA1Sqox4Of2GbaE5qEvB45wbdDMmO3M8ejoFCtt2L28I7XFCSS 0rDaRBdjV6UPLNpcnjSwjoHH/qdx0FZq+HJvsG/eT7TMH3rTPSCGkplWtLQrWmV0+suK KZXbS0PqGCfb3B79FmeWheQVItQ2tehIsRyvhRxNnBcEBrstq5hF0qKbpvxww3jvXoAH elGgmgpjR6I3D5dZtaCYKjmlELic+bPdF8cmwQ8pTQZBnODVq6ZCYYMOKdFaa4mFbr9C TK49UvC9cf6cWDWTWvtrOpLmrHatvr5j7Dd4OZjUHCa+IqUKUFr29hIrtHu9l6ErPX3x QmTg== X-Gm-Message-State: AA+aEWZMoybFSe2Gv80ea6ZM9b+9R2cjiWmu4Xl0CmXo4YqpL7SfT3FC CMBzOnlRYQOlGrhGMmt8hF2j9jHKBvdz/zeCu24/n40= X-Google-Smtp-Source: AFSGD/VO2vj029mbU7N3C/pyzVzAuv1M3o7lsUj54Yj3ILGwzsV+lxBZag0KvB2WhfuROKsNA9roMWQXqBz54j978SU= X-Received: by 2002:a24:6890:: with SMTP id v138mr6573103itb.53.1544628680406; Wed, 12 Dec 2018 07:31:20 -0800 (PST) MIME-Version: 1.0 From: Francesco Sclano Date: Wed, 12 Dec 2018 16:31:08 +0100 Message-ID: Subject: In a real cluster (not pseudo-distributed mode) composed by N datanodes eachone having X vCPU, is more appropriate to set giraph.numComputeThreads with N or with X*N ? To: user@giraph.apache.org Content-Type: multipart/alternative; boundary="000000000000ce3c3e057cd4e3e5" --000000000000ce3c3e057cd4e3e5 Content-Type: text/plain; charset="UTF-8" Hi, I'm using a cluster composed of N datanodes and I'd like to properly set following parameters: -ca giraph.numComputeThreads= -ca giraph.numInputThreads= -ca giraph.numOutputThreads= The cluster is dedicated to my giraph program (no others job run in this cluster). I'd like to know if is more appropriate to set above parameters with N * with X*N ? (where X is the number of vCPU in every datanode). p.s. As Claudio Martella says in a previous post, maybe is better to use X-1 instead of X because giraph needs 1 thread for internal work. Anyway I didn't understand if this previous post is related to a pseudo-distributed mode or a distributed mode. Hence i post this question :-P Thanks! -- Francesco Sclano --000000000000ce3c3e057cd4e3e5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,
I'm using a cluste= r composed of N datanodes and I'd like to properly set following parame= ters:

-ca giraph.numComputeThreads=3D
-ca girap= h.numInputThreads=3D
-ca giraph.numOutputThreads=3D

=
The cluster is dedicated to my giraph program (no others job run in th= is cluster).
I'd like to know if is more appropriate to s= et above parameters with N * with X*N ? (where X is the number of vCPU in e= very datanode).

p.s. As Claudio Martella says in a= previous post, maybe is better to use X-1 instead of X because giraph need= s 1 thread for internal work. Anyway I didn't understand if this previo= us post is related to a pseudo-distributed mode or a distributed mode. Henc= e i post this question :-P

Thanks!

--
Francesco Sclano
--000000000000ce3c3e057cd4e3e5--