hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3040) [Data Model] Make putEntities operation be aware of the app's context
Date Tue, 24 Mar 2015 21:28:53 GMT

    [ https://issues.apache.org/jira/browse/YARN-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378666#comment-14378666
] 

Zhijie Shen commented on YARN-3040:
-----------------------------------

Thanks for review, Sangjin and Junping! I've updated the patch accordingly.

bq. I am comfortable with continuing to work on the flow-related items in the separate JIRA.

Thanks. This sounds good.

bq. I'm not sure of these set calls. Are these here just to initialize the context to default
values? 

Yes, these are the defaults. In fact, "user" will sure to be update by the rpc call to get
the context info (unless there's bug in the RPC). The current user for initialization is usually
not correct, but I kept to have a value to ensure we have something to pass to the storage
to prevent possible NPE that will crush the process. Instead, we can easily debug/inspect
the storage to verify the user if bug occurs. I add some code comments for the initialization.

bq. I would prefer something like "yarn.cluster.id" because this id is for identifying YARN
cluster rather than ResourceManager. 

I also agree "yarn.cluster.id" sounds better, but "yarn.resourcemanager.cluster-id" is the
legacy name, which is used by RM HA for a while. As it's not sound so bad, how about keeping
it, such that we don't need to deprecate config or break compatibility.

bq. Can we add a test case that without specifying flow_id and flow_run_id and v2 timeline
service still can work?

Added the test case in the new patch

bq. Do we need to be case-insensitive here? I think we can be strict about the tag names?

This is because the tag text has case sensitive and insensitive mode. When insensitive, even
if user inputs the upper case strings, it will be normalized to lower case strings. So we
need to take care this case.

bq. You might want to be bit defensive about the tag not carrying any value (e.g. "TIMELINE_FLOW_ID_TAG:").

It will check if the tag starts with "TIMELINE_FLOW_ID_TAG:", and then if the value is empty,
{{"TIMELINE_FLOW_ID_TAG:".substring("TIMELINE_FLOW_ID_TAG".length() + 1)}} will return an
empty value. It shouldn't throw IndexOutOfBoundsException. But it seems there's no need to
add an empty env, I'll change the code accordingly.

In addition, I fixed a couple test failure in the new patch.
 

> [Data Model] Make putEntities operation be aware of the app's context
> ---------------------------------------------------------------------
>
>                 Key: YARN-3040
>                 URL: https://issues.apache.org/jira/browse/YARN-3040
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Zhijie Shen
>         Attachments: YARN-3040.1.patch, YARN-3040.2.patch, YARN-3040.3.patch
>
>
> Per design in YARN-2928, implement client-side API for handling *flows*. Frameworks should
be able to define and pass in all attributes of flows and flow runs to YARN, and they should
be passed into ATS writers.
> YARN tags were discussed as a way to handle this piece of information.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message