falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sandeep samudrala (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-2228) Falcon should also set the feed output names as a coord property
Date Mon, 02 Jan 2017 07:34:58 GMT

    [ https://issues.apache.org/jira/browse/FALCON-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15792403#comment-15792403
] 

sandeep samudrala commented on FALCON-2228:
-------------------------------------------

Post this change the configurations are passed in the following manner.
{noformat}
falconInputNames
inputData#inputData2


falconInputFeeds
feed1#feed2^M


falconInPaths
hdfs://10.14.114.50:8020/user/sandeep/testEffective/2017/01/01/05,hdfs://10.14.114.50:8020/user/sandeep/testEffective/2017/01/01/04,hdfs://10.14.114.50:8020/user/sandeep/testEffective/2017/01/01/03#hdfs://10.14.114.50:8020/user/sandeep/testEffective2/2017/01/01/05,hdfs://10.14.114.50:8020/user/sandeep/testEffective2/2017/01/01/04,hdfs://10.14.114.50:8020/user/sandeep/testEffective2/2017/01/01/03

Inputs paths are separated by "#" while output paths are separated by ","

feedInstancePaths
hdfs://10.14.114.50:8020/user/sandeep/testEffective3/2017/01/01/05,hdfs://10.14.114.50:8020/user/sandeep/testEffective4/2017/01/01/05

{noformat}

> Falcon should also set the feed output names as a coord property
> ----------------------------------------------------------------
>
>                 Key: FALCON-2228
>                 URL: https://issues.apache.org/jira/browse/FALCON-2228
>             Project: Falcon
>          Issue Type: Improvement
>            Reporter: Pallavi Rao
>            Assignee: sandeep samudrala
>
> Falcon sets the following props, in the coordinator.
>  // what inputs
>     INPUT_FEED_NAMES("falconInputFeeds", "name of the feeds which are used as inputs",
false),
>     INPUT_FEED_PATHS("falconInPaths", "comma separated input feed instance paths", false),
>     INPUT_NAMES("falconInputNames", "name of the inputs", false),
>     // what outputs
>     OUTPUT_FEED_NAMES("feedNames", "name of the feeds which are generated/replicated/deleted"),
>     OUTPUT_FEED_PATHS("feedInstancePaths", "comma separated feed instance paths"),
>   
> What is missing is the list of output names, example:
>  OUTPUT_NAMES("feedInstanceNames", "comma separated list of names of outputs", false)
> Also, Falcon must ensure that the order of the input and output names correspond with
the paths.
> This will enable a user workflow that consumes some inputs and produces some outputs
based on certain parameters, to consume these variables and use them. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message