Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6B4EB17717 for ; Mon, 15 Jun 2015 13:04:24 +0000 (UTC) Received: (qmail 83827 invoked by uid 500); 15 Jun 2015 13:04:24 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 83761 invoked by uid 500); 15 Jun 2015 13:04:24 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 83751 invoked by uid 99); 15 Jun 2015 13:04:24 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Jun 2015 13:04:24 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of tammymendt@gmail.com designates 209.85.214.172 as permitted sender) Received: from [209.85.214.172] (HELO mail-ob0-f172.google.com) (209.85.214.172) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Jun 2015 13:02:08 +0000 Received: by obcej4 with SMTP id ej4so63487170obc.0 for ; Mon, 15 Jun 2015 06:03:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=LOGLbEtGGiMp19bMkZskbV1GpbQwnnCbEYix9gR5dzU=; b=ZBXRdPMxOy+xjUFaEybgmk86JzFtAE1s2aJdyc40jWCj9KIXCd0XjBUtfd2pnuHa2Y gWMaFNBUz0iFjSMdPtEVU2MsQ+xgBbCPceCgo77BinremfLiGY2WzDMEv/p9NQ9m44D2 WqAF5qSeYT+tK6MI9LwvCDevN3kVivU0PNsfinP7tPr04d7/GN7pQ2HAWLX6s/XB7BXA vLxOPfZTxBP5kKS8CylcKNb0yOoagr+avv4OwYsJSGcOwWAklXh7+3Qz43wOfNYJfhk+ zJe35wnmh5+/EHFegI8jbWNy6A5Sb77qlGk46yfJR/3Qp2VFHjkg5WZnXJBZU/E8nigc cm1A== X-Received: by 10.202.177.68 with SMTP id a65mr22495822oif.64.1434373436222; Mon, 15 Jun 2015 06:03:56 -0700 (PDT) MIME-Version: 1.0 Received: by 10.182.66.197 with HTTP; Mon, 15 Jun 2015 06:03:35 -0700 (PDT) In-Reply-To: References: From: Tamara Mendt Date: Mon, 15 Jun 2015 15:03:35 +0200 Message-ID: Subject: Re: Monitoring memory usage of a Flink Job To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a113cd5e82434fa05188e17c3 X-Virus-Checked: Checked by ClamAV on apache.org --001a113cd5e82434fa05188e17c3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Ok great, I will try this out and get back to you. Thanks =3D) On Mon, Jun 15, 2015 at 2:52 PM, Till Rohrmann wrote= : > Hi Tamara, > > you can instruct Flink to write the current memory statistics to the log > by setting taskmanager.debug.memory.startLogThread: true in the Flink > configuration. Furthermore, you can control the logging interval with > taskmanager.debug.memory.logIntervalMs where the interval is specified in > milli seconds. > > Cheers, > Till > > On Mon, Jun 15, 2015 at 2:46 PM Fabian Hueske fhueske@gmail.com > wrote: > > Hi Tamara, >> >> what kind of information do you need? Something like, size and usage of >> in-memory sort buffers or hash tables? >> Some information might written in DEBUG logs, but I'm not sure about >> that. Besides logs, I doubt that Flink monitors memory usage. >> >> Cheers, Fabian >> >> 2015-06-15 14:34 GMT+02:00 Tamara Mendt : >> >>> Hi, >>> >>> I am running some experiments on Flink and was wondering if there is >>> some way to monitor the memory usage of a Flink Job (running locally an= d on >>> a cluster). I need to run multiple jobs and compare their memory usage. >>> >>> Cheers, >>> >>> Tamara >>> >> >> =E2=80=8B > --=20 Tamara Mendt --001a113cd5e82434fa05188e17c3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Ok great, I will try this out and get back to you. Thanks = =3D)

On = Mon, Jun 15, 2015 at 2:52 PM, Till Rohrmann <trohrmann@apache.org= > wrote:
=

Hi Tamara,

you can instruct Flink to write= the current memory statistics to the log by setting taskm= anager.debug.memory.startLogThread: true in the Flink configuration.= Furthermore, you can control the logging interval with ta= skmanager.debug.memory.logIntervalMs where the interval is specified= in milli seconds.

Cheers,
Till

On Mon, Jun 15, 2015 at 2:46 PM= Fabian Hueske fhueske@gmail.com wrote:

Hi Tamara,

wha= t kind of information do you need? Something like, size and usage of in-mem= ory sort buffers or hash tables?
Some information might written in= DEBUG logs, but I'm not sure about that. Besides logs, I doubt that Fl= ink monitors memory usage.

Cheers, Fabian

2015-06-15 14:34 GMT+02:00 = Tamara Mendt <tammymendt@gmail.com>:
Hi,

I am running some expe= riments on Flink and was wondering if there is some way to monitor the memo= ry usage of a Flink Job (running locally and on a cluster). I need to run m= ultiple jobs and compare their memory usage.
Cheers,

Tamara

=E2=80=8B



--
Tamara Mendt
--001a113cd5e82434fa05188e17c3--