hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3864) Implement support for querying single app and all apps for a flow run
Date Fri, 02 Oct 2015 19:33:26 GMT

    [ https://issues.apache.org/jira/browse/YARN-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941619#comment-14941619

Sangjin Lee commented on YARN-3864:

Thanks for updating the patch [~varun_saxena]. I'm now in transit, and it will take a little
time before I can go over the patch in more detail.

I'd like to summarize a couple of issues, and like us to tackle them after this. I don't think
they need to hold this JIRA up.

(1) the REST API
At some point, let's take a look at the REST API (as discussed earlier), and see if we need
to make them more consistent with the overall REST best practices and more importantly hadoop's
REST style (YARN, etc.). It's not terribly urgent, and we just need to review it some time

(2) storing user in the app-to-flow table
[~vrushalic] brought up the point of the user. And I'm realizing that we may need to store
the user in the app-to-flow table. Was there a reason that we didn't store the user info in
the app-to-flow table? The issue is that the user is a critical piece of the context (cluster/user/flow_id/flow_run_id/app_id),
and the app-to-flow lookup should have it.

Right now in the REST call, the user is optional, and if it is not provided it is deduced
from the caller UGI. But IMO the caller UGI is a poor/incorrect choice. This would work only
if the user/client that's executing the REST call is the owner of that YARN app. Rather, the
user should be always the user of the app, regardless of who executes the REST call (authorization
is a separate topic).

I propose that we store the user id in the app-to-flow table, and when we recover the context
on the read path, we use that user if the user is not provided by the caller. We don't have
to do it as part of this JIRA, but we should fix this shortly after. What are your thoughts?

> Implement support for querying single app and all apps for a flow run
> ---------------------------------------------------------------------
>                 Key: YARN-3864
>                 URL: https://issues.apache.org/jira/browse/YARN-3864
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>            Priority: Blocker
>         Attachments: YARN-3864-YARN-2928.01.patch, YARN-3864-YARN-2928.02.patch
> This JIRA will handle support for querying all apps for a flow run in HBase reader implementation.
> And also REST API implementation for single app and multiple apps.

This message was sent by Atlassian JIRA

View raw message