Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 356A317924 for ; Tue, 24 Feb 2015 10:41:56 +0000 (UTC) Received: (qmail 95402 invoked by uid 500); 24 Feb 2015 10:41:50 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 95252 invoked by uid 500); 24 Feb 2015 10:41:50 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 95242 invoked by uid 99); 24 Feb 2015 10:41:49 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Feb 2015 10:41:49 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of anytek88@gmail.com designates 209.85.223.173 as permitted sender) Received: from [209.85.223.173] (HELO mail-ie0-f173.google.com) (209.85.223.173) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Feb 2015 10:41:44 +0000 Received: by iecat20 with SMTP id at20so30832619iec.12 for ; Tue, 24 Feb 2015 02:41:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=p0f4YOUdoOmCLrgWDMCCDBUJvXlE/IiYut3kaCq+nsw=; b=kceN1cPioJPAEKAfg10a908jUvYfR7Um7Vt6fYe5k9/EWm/kFJywR7BymxMJ78YjzD kM2Z5acJ0+iiFRNiJaIk6TXyacL0jQhtknC1ItM47ssI2KU+0cZCEyROJaKlNJZ5hBAa f/Lu8L9YqiGfsX+pFLhfqf3xjTZ96/eQX0fudMaiURy1IC3OYRS2hKM4m+RZTCsB9l24 ttWQo9Wz1umFlK54IolZhKdzBxYGk/XJUwFE0ZkIpKqcV7jzkM+4y30aGakDKFsBMDLs MN3Odn56IA/1psByi6x3fylFe5b8PIHkIn7vuYANozek16OSfFiaAqc5unL/7oAfAFoL 2hoQ== MIME-Version: 1.0 X-Received: by 10.43.131.5 with SMTP id ho5mr3876918icc.82.1424774483412; Tue, 24 Feb 2015 02:41:23 -0800 (PST) Received: by 10.36.37.1 with HTTP; Tue, 24 Feb 2015 02:41:23 -0800 (PST) Date: Tue, 24 Feb 2015 11:41:23 +0100 Message-ID: Subject: Can RM ignore heartbeats? From: "Fabio C." To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=20cf307f30baf8104a050fd328c5 X-Virus-Checked: Checked by ClamAV on apache.org --20cf307f30baf8104a050fd328c5 Content-Type: text/plain; charset=UTF-8 Hi everyone, I have a question about the ResourceManager behavior: when the ResourceManager allocates a container, it takes some time before the NMToken is sent and then received by the ApplicationMaster. During this time, it is possible to receive another heartbeat from the AM, equal to the last one (since the AM is not aware of the allocated resources). Is there any policy in YARN that makes the RM aware of this and ignore this last heartbeat? I ask this because I would expect way more superfluous containers allocated, in comparison to the ones I can see from the logs. Thanks in advance Fabio --20cf307f30baf8104a050fd328c5 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hi everyone,
I have a question about the Reso= urceManager behavior:
when the ResourceManager allocates a container, it= takes some time before the NMToken is sent and then received by the Applic= ationMaster.
During this time, it is possible to receive another heartb= eat from the AM, equal to the last one (since the AM is not aware of the al= located resources).
Is there any policy in YARN that makes the RM aware = of this and ignore this last heartbeat?
I ask this because I would expec= t way more superfluous containers allocated, in comparison to the ones I ca= n see from the logs.

Thanks in advance

Fabio
<= /div> --20cf307f30baf8104a050fd328c5--