Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7BDEE19394 for ; Wed, 27 Apr 2016 23:12:18 +0000 (UTC) Received: (qmail 30311 invoked by uid 500); 27 Apr 2016 23:12:13 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 30248 invoked by uid 500); 27 Apr 2016 23:12:13 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 30193 invoked by uid 99); 27 Apr 2016 23:12:13 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Apr 2016 23:12:13 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id E8C102C1F61 for ; Wed, 27 Apr 2016 23:12:12 +0000 (UTC) Date: Wed, 27 Apr 2016 23:12:12 +0000 (UTC) From: "Ram Venkatesh (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-4913) Yarn logs should take a -out option to write to a directory MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261164#comment-15261164 ] Ram Venkatesh commented on YARN-4913: ------------------------------------- [~xgong] Here are two reasons for the -out option, both are more relevant for large multi-GB app instances. 1. yarn logs > $targetFile produces a single file that appends all the individual container logs. This requires (clumsy | complex) parsing to split the files apart if you are looking for specific task or app-specific logs. Writing to a directory will preserve the distinct files easily and also lend itself to archiving. 2. redirecting through the console instead of writing directly to the local filesystem apis adds additional overhead on some platforms like Windows. >From a supportability standpoint I think this option will be useful. > Yarn logs should take a -out option to write to a directory > ----------------------------------------------------------- > > Key: YARN-4913 > URL: https://issues.apache.org/jira/browse/YARN-4913 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Xuan Gong > Assignee: Xuan Gong > Attachments: YARN-4913.1.patch, YARN-4913.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)