Return-Path: X-Original-To: apmail-flink-dev-archive@www.apache.org Delivered-To: apmail-flink-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 848D917673 for ; Wed, 8 Apr 2015 10:34:42 +0000 (UTC) Received: (qmail 22688 invoked by uid 500); 8 Apr 2015 10:33:44 -0000 Delivered-To: apmail-flink-dev-archive@flink.apache.org Received: (qmail 22620 invoked by uid 500); 8 Apr 2015 10:33:43 -0000 Mailing-List: contact dev-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list dev@flink.apache.org Received: (qmail 22600 invoked by uid 99); 8 Apr 2015 10:33:43 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Apr 2015 10:33:43 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of till.rohrmann@gmail.com designates 209.85.220.170 as permitted sender) Received: from [209.85.220.170] (HELO mail-qk0-f170.google.com) (209.85.220.170) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Apr 2015 10:33:38 +0000 Received: by qkx62 with SMTP id 62so79948488qkx.0 for ; Wed, 08 Apr 2015 03:31:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=fkF902SeUJqSMdCY7WUa2if1WCh+nIAG+qQEa2luetI=; b=H8CuGkDnFnQBGS8K2aCSoj6+rTsbem50aWUMvxBsDW5hrrpDzmfqKRxJRtCtvIwkwL OI7vRsDkbtHBu59UilrJKeQ+ccXbPnHYkQU3sSv0XOoRxM3NLfybWz4dHk/a3rxeC/jl KywUYVynWRs117ox3wjCVIF5OG+Z4YIP0sEWuJg9DXmdozd0lI+pUROU0SQzlQteDW2B o4vmaNs+3y4uG/i2pHbEdbpGrVFeCQO3Gu4kzFOKyhvn4bDU26TDyuqzR2wwyduP5rlO KbUYZ5VGYo3ARM74DrO0o1wgb90dDxmKvOo+SYINaljuja5V5+afeo61UxlO0neOoQk8 qZMg== MIME-Version: 1.0 X-Received: by 10.55.31.168 with SMTP id n40mr48068722qkh.56.1428489062614; Wed, 08 Apr 2015 03:31:02 -0700 (PDT) Received: by 10.229.53.202 with HTTP; Wed, 8 Apr 2015 03:31:02 -0700 (PDT) In-Reply-To: References: Date: Wed, 8 Apr 2015 12:31:02 +0200 Message-ID: Subject: Re: [jira] [Created] (FLINK-1843) Job History gets cleared too fast From: Till Rohrmann To: dev@flink.apache.org Content-Type: multipart/alternative; boundary=001a11479130248c2705133407b5 X-Virus-Checked: Checked by ClamAV on apache.org --001a11479130248c2705133407b5 Content-Type: text/plain; charset=UTF-8 This also happens for cluster setups. On Wed, Apr 8, 2015 at 11:29 AM, Maximilian Michels (JIRA) wrote: > Maximilian Michels created FLINK-1843: > ----------------------------------------- > > Summary: Job History gets cleared too fast > Key: FLINK-1843 > URL: https://issues.apache.org/jira/browse/FLINK-1843 > Project: Flink > Issue Type: Bug > Components: JobManager > Affects Versions: 0.9 > Reporter: Maximilian Michels > Fix For: 0.9 > > > As per FLINK-1442, the JobManager stores the archived ExecutionGraph > behind a SoftReference. At least for local setups, this mechanism doesn't > seem to work properly. There are two issues: > > - The history gets cleared too fast > - The history gets cleared in a non-sequential fashion, i.e. arbitrary old > ExecutionGraph are discarded > > To solve these problems we might > > - Store the least recent ExecutionGraph behind a SoftReference > - Store the most recent ExecutionGraphs without a SoftReference > > That way, we can save memory but have the latest history available to the > user. We might introduce a configuration variable where the user can > specify the number of ExecutionGraphs that should be held in memory. The > remaining can be stored behind a SoftReference. > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332) > --001a11479130248c2705133407b5--