Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 189C117ACB for ; Mon, 10 Nov 2014 13:14:35 +0000 (UTC) Received: (qmail 95849 invoked by uid 500); 10 Nov 2014 13:14:35 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 95808 invoked by uid 500); 10 Nov 2014 13:14:34 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 95795 invoked by uid 99); 10 Nov 2014 13:14:34 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Nov 2014 13:14:34 +0000 Date: Mon, 10 Nov 2014 13:14:34 +0000 (UTC) From: "Naganarasimha G R (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-2808) yarn client tool can not list app_attempt's container info correctly MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14204747#comment-14204747 ] Naganarasimha G R commented on YARN-2808: ----------------------------------------- "historyClient" in YarnClientImpl will behave differently for getContainers call for a running application. When deprecated AHS is configured it throws exception and timeline history server is configured all containers result is got of which completed containers can be utilized. So based on which server is configured, YarnClientImpl need to take decision whether containers for running application needs to be considered from history client. hence this issue depends on yarn-2838 > yarn client tool can not list app_attempt's container info correctly > -------------------------------------------------------------------- > > Key: YARN-2808 > URL: https://issues.apache.org/jira/browse/YARN-2808 > Project: Hadoop YARN > Issue Type: Bug > Components: client > Reporter: Gordon Wang > Assignee: Naganarasimha G R > > When enabling timeline server, yarn client can not list the container info for a application attempt correctly. > Here is the reproduce step. > # enabling yarn timeline server > # submit a MR job > # after the job is finished. use yarn client to list the container info of the app attempt. > Then, since the RM has cached the application's attempt info, the output show > {noformat} > [hadoop@localhost hadoop-3.0.0-SNAPSHOT]$ ./bin/yarn container -list appattempt_1415168250217_0001_000001 > 14/11/05 01:19:15 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable > 14/11/05 01:19:15 INFO impl.TimelineClientImpl: Timeline service address: http://0.0.0.0:8188/ws/v1/timeline/ > 14/11/05 01:19:16 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 > 14/11/05 01:19:16 INFO client.AHSProxy: Connecting to Application History server at /0.0.0.0:10200 > Total number of containers :0 > Container-Id Start Time Finish Time State Host LOG-URL > {noformat} > But if the rm is restarted, client can fetch the container info from timeline server correctly. > {noformat} > [hadoop@localhost hadoop-3.0.0-SNAPSHOT]$ ./bin/yarn container -list appattempt_1415168250217_0001_000001 > 14/11/05 01:21:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable > 14/11/05 01:21:06 INFO impl.TimelineClientImpl: Timeline service address: http://0.0.0.0:8188/ws/v1/timeline/ > 14/11/05 01:21:06 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 > 14/11/05 01:21:06 INFO client.AHSProxy: Connecting to Application History server at /0.0.0.0:10200 > Total number of containers :4 > Container-Id Start Time Finish Time State Host LOG-URL > container_1415168250217_0001_01_000001 1415168318376 1415168349896 COMPLETE localhost.localdomain:47024 http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_000001/container_1415168250217_0001_01_000001/hadoop > container_1415168250217_0001_01_000002 1415168326399 1415168334858 COMPLETE localhost.localdomain:47024 http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_000002/container_1415168250217_0001_01_000002/hadoop > container_1415168250217_0001_01_000003 1415168326400 1415168335277 COMPLETE localhost.localdomain:47024 http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_000003/container_1415168250217_0001_01_000003/hadoop > container_1415168250217_0001_01_000004 1415168335825 1415168343873 COMPLETE localhost.localdomain:47024 http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_000004/container_1415168250217_0001_01_000004/hadoop > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)