Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 20 Oct 2016 18:45:59 +0000 (UTC)
From: "Joep Rottinghuis (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13012831.1476703592000.42889.1476989159265@Atlassian.JIRA>
In-Reply-To: <JIRA.13012831.1476703592000@Atlassian.JIRA>
References: <JIRA.13012831.1476703592000@Atlassian.JIRA> <JIRA.13012831.1476703592763@arcas>
Subject: [jira] [Commented] (YARN-5742) Serve aggregated logs of historical
 apps from timeline service
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 20 Oct 2016 18:46:03 -0000


    [ https://issues.apache.org/jira/browse/YARN-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15592635#comment-15592635 ] 

Joep Rottinghuis commented on YARN-5742:
----------------------------------------

We should really carefully consider whether serving yarn application logs is a timeline service concern.
I'd argue that belongs in a separate Yarn service. It would be perfectly acceptable to have that separate service store metadata (about the current log location of tasks, whether that is on local disk, or aggregated to a location such as HDFS) in the timeline service, but I think the serving itself doesn't below here.

Providing an api that can read files from HDFS and stream them out would be opening security concerns, would duplicate efforts from services such as WebHDFS/HttpFS and all their concerns. The HttpFS approach of a central pool of nodes serving data from HDFS has been superseded by a distributed WebHDFS approach. Note by the way that WebHDFS as it stands today still has some compatibility challenges with HDFS federation. Both of these general approaches of serving HDFS data have to deal with proxying user requests and correctly limiting visibility of HDFS files to the users with the appropriate access.

> Serve aggregated logs of historical apps from timeline service
> --------------------------------------------------------------
>
>                 Key: YARN-5742
>                 URL: https://issues.apache.org/jira/browse/YARN-5742
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Varun Saxena
>            Assignee: Rohith Sharma K S
>         Attachments: YARN-5742-POC-v0.patch
>
>


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org