hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5647) [Security] Collector and reader side changes for loading auth filters and principals
Date Mon, 19 Dec 2016 19:19:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15762056#comment-15762056

Sangjin Lee commented on YARN-5647:

As of now, we will have one level db storage per NM (per NM Collector Manager). This, however,
would need a change once collector (not collector manager) itself is launched as a separate
process i.e. as a system container if we ever do it. We need to see how to handle failures
for such a container. And that would require more discussion.
Anyways token recovery related stuff will be handled in a separate JIRA. So we can have more
discussion in detail there once that is opened.

Thanks for the clarification [~varun_saxena]. I understand that having timeline collectors
as out-of-process containers would be different in this regard. I think it is acceptable to
capture that as part of the container work.

I also think having multiple reader instances might complicate things here. Today it is still
a daemon, so there would be a single instance per cluster, so we might be OK with using a
local LevelDB attached to it.

Something to think about and answer, however: how do we ensure we don't get mixed up with
the timeline state store from v.1 on the same node? Is it not a concern because in v.1 the
state store would be on the RM machine? But that would also mean we need to be careful with
running the reader on the RM machine? Should we at least see if we can differentiate the v.1
timeline state store and v.2 so they don't get mixed up?

> [Security] Collector and reader side changes for loading auth filters and principals
> ------------------------------------------------------------------------------------
>                 Key: YARN-5647
>                 URL: https://issues.apache.org/jira/browse/YARN-5647
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>              Labels: yarn-5355-merge-blocker
>         Attachments: YARN-5647-YARN-5355.wip.002.patch, YARN-5647-YARN-5355.wip.003.patch,

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message