hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3391) Clearly define flow ID/ flow run / flow version in API and storage
Date Wed, 08 Apr 2015 13:37:14 GMT

    [ https://issues.apache.org/jira/browse/YARN-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14485237#comment-14485237
] 

Junping Du commented on YARN-3391:
----------------------------------

Thanks [~zjshen] for updating the patch!
bq. According to Sangjin's given example, we usually want to identify a flow run by timestamp,
which theoretically can be negative to represent sometime before 1970.
Except time travel, I don't believe any flow run running on hadoop and new timeline service
should happen before 1970. :) 
Anyway, we do have some practice to check timestamp > 0 (like: MetricsRecordImpl), but
more cases sounds like we didn't do this negative check for timestamp. Given this, I am fine
with not checking here.

v4 patch looks good to me. [~sjlee0], [~vrushalic] and [~jrottinghuis], any additional comments
for the patch?

> Clearly define flow ID/ flow run / flow version in API and storage
> ------------------------------------------------------------------
>
>                 Key: YARN-3391
>                 URL: https://issues.apache.org/jira/browse/YARN-3391
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>         Attachments: YARN-3391.1.patch, YARN-3391.2.patch, YARN-3391.3.patch, YARN-3391.4.patch
>
>
> To continue the discussion in YARN-3040, let's figure out the best way to describe the
flow.
> Some key issues that we need to conclude on:
> - How do we include the flow version in the context so that it gets passed into the collector
and to the storage eventually?
> - Flow run id should be a number as opposed to a generic string?
> - Default behavior for the flow run id if it is missing (i.e. client did not set it)
> - How do we handle flow attributes in case of nested levels of flows?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message