Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id BE788200C25 for ; Fri, 24 Feb 2017 23:59:51 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id BD201160B79; Fri, 24 Feb 2017 22:59:51 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E02DE160B62 for ; Fri, 24 Feb 2017 23:59:50 +0100 (CET) Received: (qmail 60390 invoked by uid 500); 24 Feb 2017 22:59:49 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 60380 invoked by uid 99); 24 Feb 2017 22:59:49 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Feb 2017 22:59:49 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id EBA51C2458 for ; Fri, 24 Feb 2017 22:59:48 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.379 X-Spam-Level: ** X-Spam-Status: No, score=2.379 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id Kequffz4U9WX for ; Fri, 24 Feb 2017 22:59:48 +0000 (UTC) Received: from mail-vk0-f49.google.com (mail-vk0-f49.google.com [209.85.213.49]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id DD0AB5FAFB for ; Fri, 24 Feb 2017 22:59:47 +0000 (UTC) Received: by mail-vk0-f49.google.com with SMTP id r136so19734488vke.1 for ; Fri, 24 Feb 2017 14:59:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=FQ84Ypv9/DWlVSaqpmEh/lGQuyFiX8GCxJpGaQSPG1c=; b=M3qHIGfMVUd8JVG6P1tFYYTinpLhdy36FJ45RbfFQmY5bAtsAcwkxTVU+CF80ahzr3 XR5TUMuJ696eeB4Xv4fEIxiRwd0dnZY+1b4jMr5y84pfCYW9DJnqL3FsNGdaHXCTo+oE dya2P4IBKUaICGofZRWNHv6HzqOAqsJ/3IxPLSimpk8HBOzqjhwnggU9T35hV1bRNq8E qEcwWb/qg5QjPefSIBqmwlW7p5rjbLif8nXPAXnu6rDbTiGbRyxrtbMSWYn1uwehuG8k 09IM820pAt76FOEL5t+PbHGnLNlISMXHOarqZLlLP5JNdV/ayX5starFljQQ16ZCjVtN oL0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=FQ84Ypv9/DWlVSaqpmEh/lGQuyFiX8GCxJpGaQSPG1c=; b=XYdurMEfj0NuUnoUzN9bepCvQynlTdLvqlGo59ScseV0W5/fzdS2B7LdM6VK7A46ff yqBVgiZl+PlqropV7Ezk8mqBsJJdDPXyLVA8VjlcNAFuYDpSgMOxw+wF1/qQzOe9o43T 3euRNIZDIUHIOXE/iWojAdT5G9KqFR2ss3C9B0FvKiaTaWrY247mvi+9H4UtPtxSkIp1 Fs2HFGDdFyFfxMsgYAhLWKvZs4xnJzl/QeVIKhtQDEpTIsRlrE5ah1SmdfZvfNy6k/0h jr2opnZixaZs1NPC5dvHhVKxDEm0rmPByNkj826R7OpPG+xShfk4aK0tpoWl+KXASw7s UgiA== X-Gm-Message-State: AMke39le96Fizw/PN42BK9vD0PR8YFdJxaArdd49U1glUyFRZwPfN3rNG7aQE83YBoAueZGQGAOWRtOHIOb4Vg== X-Received: by 10.31.70.66 with SMTP id t63mr1751479vka.19.1487977187115; Fri, 24 Feb 2017 14:59:47 -0800 (PST) MIME-Version: 1.0 Received: by 10.159.41.102 with HTTP; Fri, 24 Feb 2017 14:59:46 -0800 (PST) In-Reply-To: References: From: Mohit Anchlia Date: Fri, 24 Feb 2017 14:59:46 -0800 Message-ID: Subject: Re: Java 7 -> 8 - Association failed error To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a11484ae0ac0a3005494eaf60 archived-at: Fri, 24 Feb 2017 22:59:51 -0000 --001a11484ae0ac0a3005494eaf60 Content-Type: text/plain; charset=UTF-8 This issue seem to match what I am observing. I ended up changing yarn.nodemanager.vmem-check-enabled to false. Is this a bug? is there a side effect? On Fri, Feb 24, 2017 at 2:54 PM, Ted Yu wrote: > Can you give more detail ? > > I found: > https://issues.apache.org/jira/browse/YARN-4714 > > Cheers > > On Fri, Feb 24, 2017 at 2:41 PM, Mohit Anchlia > wrote: > >> Figured out. It looks like there is a virtual memory limit check enforced >> in yarn which just surfaced with java 8 >> >> On Fri, Feb 24, 2017 at 2:09 PM, Mohit Anchlia >> wrote: >> >>> I recently upgraded the cluster from java 7 to java 8. Now when I run >>> flink on a yarn cluster I see errors: Eventually application gives up and >>> terminates. Any suggestions? >>> >>> Association with remote system [akka.tcp://flink@slave:35543] has >>> failed, address is now gated for [5000] ms. Reason: [Disassociated] >>> >> >> > --001a11484ae0ac0a3005494eaf60 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
This issue seem to match what I am observing. I ended up c= hanging yarn.= nodemanager.vmem-check-enabled to false. Is this a bug? is there a side eff= ect?

= On Fri, Feb 24, 2017 at 2:54 PM, Ted Yu <yuzhihong@gmail.com> wrote:
Can you give = more detail ?


Cheer= s

On Fri, Feb 24, 2017 at 2:41 PM, Mohit = Anchlia <mohitanchlia@gmail.com> wrote:
Figured out. It looks like there is a virtual memory = limit check enforced in yarn which just surfaced with java 8

On Fri, Feb 24, 201= 7 at 2:09 PM, Mohit Anchlia <mohitanchlia@gmail.com> wr= ote:
I recently upgraded the clus= ter from java 7 to java 8. Now when I run flink on a yarn cluster I see err= ors: Eventually application gives up and terminates. Any suggestions?
=

Association with remote system [akka.tcp://flink@slave:= 35543] has failed, address is now gated for [5000] ms. Reason: [Disassociat= ed]



--001a11484ae0ac0a3005494eaf60--