Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5AA9B191F1 for ; Mon, 4 Apr 2016 13:54:16 +0000 (UTC) Received: (qmail 19962 invoked by uid 500); 4 Apr 2016 13:54:15 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 19865 invoked by uid 500); 4 Apr 2016 13:54:15 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 19854 invoked by uid 99); 4 Apr 2016 13:54:15 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Apr 2016 13:54:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 32B65C05EF for ; Mon, 4 Apr 2016 13:54:15 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.821 X-Spam-Level: X-Spam-Status: No, score=-0.821 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id rcF-6AqocK_G for ; Mon, 4 Apr 2016 13:54:14 +0000 (UTC) Received: from mail-pa0-f49.google.com (mail-pa0-f49.google.com [209.85.220.49]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id EA5F75FB6F for ; Mon, 4 Apr 2016 13:54:13 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id fe3so144916009pab.1 for ; Mon, 04 Apr 2016 06:54:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to; bh=LpvEF2y6XalzrUJG0O9VKzWZom5hMPCzrWWbhJUtuDU=; b=UGNdJN91c5xQluKS84AU2+DUMewQzrsOoJobapitgaeL35BpdDQGnjIms+b4l79XXq 56/uzZ0jMHyQZUbbrkiOwi58KRpHB4BfDXGHG+1kZZavVGqn/LFRdbHCl/2vOh0tNUAU HbxyWjJzbT4pqrc6du8KEKFmXBl5YwTUgGPRbD5dD93l469EIpJuLBuDy+7s+sqammhW 5QyYLUmjA0uPFFMXpFIFBm9R5jLriyuMYB3PY8rl/gyCEtW4c4tEE/Mc516IE//oo1/K /VmLCP9ubWEpjuVIA8YqoV006thtz76PbSdlanbFxp2UeBJllfKDwwAW5lLhLdwDBvPy SL2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to; bh=LpvEF2y6XalzrUJG0O9VKzWZom5hMPCzrWWbhJUtuDU=; b=FTJp5XLzOXiElEGXIbvhS5eFXR2Wy4jnbLjNHTNnPdyPZRcp3eSbSofqC55/6JCgGK tIWPtcm2yjoQzL/XgSsWy99kIf4GGZ/4YyAQaCPhqyVRO+E7Vx61EQMVXK5FR46h8wU/ 4XTVULBGw6W2T+r5+QqL+DLehOz1wKUgpQ0nI3IwAFHpFpj1kvQcE6egAarQr9sWIC31 KV4Gb8Oc31YcQgLiBGN2cBoywVMVIyHm4lH56JgX/FOg9qdLuxeiWFQX0YPhqSt+cvBw hJDKor8HbD//Kk0bIU5TRYDYVkmv96MbZQN0/yloJfiBm7IXSSdM3+PWTACICNE+Er+z Cu5g== X-Gm-Message-State: AD7BkJLciF3tMMD3LLlWdLlf7vu2pSKzAvSgDiNBhcfwK8YSiKc5K+UQp/T/XgsfXSmGXphUBOJMV5+SJiSC4A== MIME-Version: 1.0 X-Received: by 10.67.30.163 with SMTP id kf3mr53775592pad.45.1459778047453; Mon, 04 Apr 2016 06:54:07 -0700 (PDT) Received: by 10.66.159.38 with HTTP; Mon, 4 Apr 2016 06:54:07 -0700 (PDT) In-Reply-To: References: Date: Mon, 4 Apr 2016 22:54:07 +0900 Message-ID: Subject: Re: scaling a flink streaming application on a single node From: Shinhyung Yang To: user@flink.apache.org Content-Type: text/plain; charset=UTF-8 Dear Aljoscha and Ufuk, Thank you for clarifying! Yes I'm running this wordcount application on a 64-core machine with 120GB ram allocated for users. > In that case, the amount of RAM you give to the TaskManager seems to low. > Could you try re-running your experiments with: > jobmanager.heap.mb: 5000 > taskmanager.heap.mb: 100000 > > So, about 100 GB of RAM for the TaskManager. Definitely I will try this! The result will be really interesting for sure. In this case, am I still good to go with 64 task slots with a single task manager? Thank you. With best regards, Shinhyung Yang.