Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D1E87200D26 for ; Fri, 20 Oct 2017 12:55:51 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id D05B8160BCB; Fri, 20 Oct 2017 10:55:51 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1F1A61609E1 for ; Fri, 20 Oct 2017 12:55:50 +0200 (CEST) Received: (qmail 13694 invoked by uid 500); 20 Oct 2017 10:55:49 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 13684 invoked by uid 99); 20 Oct 2017 10:55:49 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Oct 2017 10:55:49 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 852D01A117F for ; Fri, 20 Oct 2017 10:55:47 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id 4oojVnoQz7eD for ; Fri, 20 Oct 2017 10:55:46 +0000 (UTC) Received: from mail-wm0-f41.google.com (mail-wm0-f41.google.com [74.125.82.41]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 2D4BA5FCD4 for ; Fri, 20 Oct 2017 10:55:46 +0000 (UTC) Received: by mail-wm0-f41.google.com with SMTP id u138so21858838wmu.4 for ; Fri, 20 Oct 2017 03:55:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=zH8VJrjO+JDw87tHcZMkKXQ/2C3XZ03+/rdIiGQn5HI=; b=BGVCY2vCcd1fn7XEyTpcodzdCebBknXf1cWuKcbqWo2MT5x6NUbN8U0pHl2QkNkbiB 4vNJsDmZRhEKK1r+hpXgk3ub5YkQBBC+cZOcc43DxJjdY9elyBnNZ84u54TJD4sOpWfd ATNssrX8xEncVIfdfCbWbH/Y745pzNgjuaRuYV4r7Hs0fMOcBA3ziM+ICgoLDthfV1za yLDGspqGmnmtfGQaJ9mOHLASNYF+SqH+BpV0pdfvtOnKZo9hlhf4uCAeH2QcSx/P8Aho iTa5oGZRAd4ir05Wr1xSQSq3iHv4jQyt6H4l5hx/wAQFYEdEU2F8loTJwzdhwe0T0MaR 9t/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=zH8VJrjO+JDw87tHcZMkKXQ/2C3XZ03+/rdIiGQn5HI=; b=DAMnQaKIqtFbP+XUnYh71yfTgzMpCoYyZM/7Mqz9sD3jHRWApB8UZwsoxDmqKUfpZy swEcuarxlKsuPgs6m+WLqQylsn0g8lv6m4+D0JCoE2sfG1el1AvWWge7LC1szr+Jlyms Pk9s/JM0aUOw6eNQg9hjFyY2ycpycLkZV3pILXrx3GNEiqLENotkhA1tjUHlpKst/tBq vdliHi//IEu7QFM4fHQve3gTwWwmpKDC/+oc80Cz4+oU6oXnns7eV3F4fyTQBGT3shKl dsjCrZj+MfQFFkdcpb4PMZhjh78DAjkpqPYc8skEl0xTpLeM+zv+TJmycXl/I27tRmSO 3O6A== X-Gm-Message-State: AMCzsaVQc6YfdcZ3TOs2mmIFWplMKcfot/5ZYicv92k0D5cGEzLw7fLd ibXB9JD8C5GGRHcqTAp92buMDfXJTBPM3C2nvC8= X-Google-Smtp-Source: ABhQp+RAtV5Nq6JEpkgjZ12ZfOHwS0UA+i/0QT+9PTKI5ni+Rv9kpUCtKZz9p4QXx4cD4dLPx3Ibq7KNJyyZ64VTGcg= X-Received: by 10.28.238.193 with SMTP id j62mr1338246wmi.44.1508496939252; Fri, 20 Oct 2017 03:55:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.148.68 with HTTP; Fri, 20 Oct 2017 03:55:38 -0700 (PDT) From: Rahul Raj Date: Fri, 20 Oct 2017 16:25:38 +0530 Message-ID: Subject: Does heap memory gets released after computing windows? To: user Content-Type: multipart/alternative; boundary="001a114705c03582c6055bf85032" archived-at: Fri, 20 Oct 2017 10:55:52 -0000 --001a114705c03582c6055bf85032 Content-Type: text/plain; charset="UTF-8" Hi All, I have been facing "GC Overhead limit exceeded" exception for quite a time with my flink program. In my flink program, I have created a session window with time gap of 10 minutes and I am performing some custom reduce operation using Window function. The current statebackend is File System. The program works fine for 15-20 mins and then fails with "GC overhead limited exception" . My understanding of flink is that, the task manager's memory will be released as soon as operation under window function will be completed. But then why I am getting this error? What should I do to avoid this? Rahul Raj --001a114705c03582c6055bf85032 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi All,

I have been facing "GC Ove= rhead limit exceeded" exception for quite a time with my flink program= . In my flink program, I have created a session window with time gap of 10 = minutes and I am performing some custom reduce operation using Window funct= ion. The current statebackend is File System. The program works fine for 15= -20 mins and then fails with "GC overhead limited exception" . My= understanding of flink is that, the task manager's memory will be rele= ased as soon as operation under window function will be completed. But then= why I am getting this error? What should I do to avoid this?
Rahul Raj
--001a114705c03582c6055bf85032--