Mailing-List: contact dev-help@falcon.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@falcon.incubator.apache.org
Date: Thu, 17 Jul 2014 06:11:04 +0000 (UTC)
From: "Angad Singh (JIRA)" <jira@apache.org>
To: dev@falcon.incubator.apache.org
Message-ID: <JIRA.12727852.1405575317666.60279.1405577464873@arcas>
In-Reply-To: <JIRA.12727852.1405575317666@arcas>
References: <JIRA.12727852.1405575317666@arcas>
Subject: [jira] [Commented] (FALCON-510) Inject falcon related properties to
 job conf
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/FALCON-510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14064620#comment-14064620 ] 

Angad Singh commented on FALCON-510:
------------------------------------

Another required property: frequency (to optimize hraven's graphite storage for a job based on its granularity instead of a generic). In general, all process properties being injected will be optimal.

> Inject falcon related properties to job conf
> --------------------------------------------
>
>                 Key: FALCON-510
>                 URL: https://issues.apache.org/jira/browse/FALCON-510
>             Project: Falcon
>          Issue Type: Improvement
>            Reporter: Shwetha G S
>
> Currently there is no falcon context injected at MR job level. The job conf has at most the oozie workflow / action ID either in the job name or sometimes in the job conf.
> Therefore there is no way for a tool like hraven, which relies completely on jobconf and job history data, to identify that a particular job maps to a particular falcon process or it's instance time, etc. Right now hraven does regex-based job name surgery on a best effort basis before emitting metrics to graphite
> Request the following feature in falcon:
> Add the following properties to the job conf (for all jobs - be it a pig action or an MR action):
> falcon.process.name
> falcon.process.instancetime
> while we're at it, might as well add any other falcon context as a jobconf property (like whether it was a rerun or the input/output feeds, cluster, validity, any process properties, etc.)
> This will ofcourse inject at the first job level and cannot ensure that any child jobs get the properties passed on (unless we can figure out a way to do that too).


--
This message was sent by Atlassian JIRA
(v6.2#6252)