falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha G S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-510) Inject falcon related properties to job conf
Date Mon, 13 Oct 2014 11:50:34 GMT

    [ https://issues.apache.org/jira/browse/FALCON-510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169205#comment-14169205
] 

Shwetha G S commented on FALCON-510:
------------------------------------

Till then, can we modify the user workflow and add the global config?

> Inject falcon related properties to job conf
> --------------------------------------------
>
>                 Key: FALCON-510
>                 URL: https://issues.apache.org/jira/browse/FALCON-510
>             Project: Falcon
>          Issue Type: Improvement
>            Reporter: Shwetha G S
>            Assignee: Peeyush Bishnoi
>         Attachments: falcon-510.txt
>
>
> Currently there is no falcon context injected at MR job level. The job conf has at most
the oozie workflow / action ID either in the job name or sometimes in the job conf.
> Therefore there is no way for a tool like hraven, which relies completely on jobconf
and job history data, to identify that a particular job maps to a particular falcon process
or it's instance time, etc. Right now hraven does regex-based job name surgery on a best effort
basis before emitting metrics to graphite
> Request the following feature in falcon:
> Add the following properties to the job conf (for all jobs - be it a pig action or an
MR action):
> falcon.process.name
> falcon.process.instancetime
> while we're at it, might as well add any other falcon context as a jobconf property (like
whether it was a rerun or the input/output feeds, cluster, validity, any process properties,
etc.)
> This will ofcourse inject at the first job level and cannot ensure that any child jobs
get the properties passed on (unless we can figure out a way to do that too).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message