falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Balu Vellanki <bvella...@hortonworks.com>
Subject Re: [DISCUSS] Lifecycle design closure
Date Wed, 09 Sep 2015 04:53:56 GMT
This is https://issues.apache.org/jira/browse/FALCON-965

On 9/8/15, 11:29 AM, "Seetharam Venkatesh" <venkatesh@innerzeal.com> wrote:

>It would help to add the Jira link and/or paste the design document here.
>
>On Tue, Sep 8, 2015 at 11:27 AM Ajay Yadav <ajaynsit@gmail.com> wrote:
>
>> Hello everyone,
>>
>> During our previous sync up we discussed feed lifecycle. I have
>> incorporated the suggestions in the updated design doc (attached). In
>>this
>> email I will try to answer the remaining unresolved questions.
>>
>> *Target Audience?*
>> Falcon users are the people who write falcon processes and feeds and use
>> Falcon CLI/ Rest API to maintain/operate them. Falcon developers are the
>> people who work on enhancing falcon, regardless of the their decision to
>> contribute it back to the Apache Falcon, though it is highly encouraged.
>> Lifecycle extensions are meant for falcon developers to extend.
>>
>> Through the lifecycle feature we aim to provide clean extension points
>>for
>> extending feed management. It will be easier to understand falcon feed
>> lifecycle with the analogy of oozie-el-extensions.
>>
>>
>> *Is a restart of falcon required for adding a new lifecycle policy?*
>> Yes, a restart is required. Lifecycle policies are applicable for all
>>feeds
>> in the falcon and we want addition of new lifecycle policies to be an
>> explicit and authorised action. Moreover, addition of new policy for a
>> lifecycle stage is a rare requirement and shouldn't be a hindrance in
>>it's
>> adoption.
>>
>>
>> *Will lifecycle policies run as a separate process?*
>> No. Execution of lifecycle policies happen outside of falcon. Some parts
>> like preparing the workflows and validation are done inside falcon at
>>the
>> time of submission of a feed. Again with the analogy of
>> oozie-el-extensions, this is an extension of falcon and is different
>>from
>> user submitted workflows/process definitions and running it in a
>>separate
>> JVM will be an overkill.
>>
>>
>> I have attached the latest doc and xsd patch in the JIRA for your
>> consideration. Would like to hear your thoughts. Once we have closure
>>over
>> design, I will work towards providing a complete patch for base
>>framework.
>>
>>
>> Cheers
>> Ajay Yadava
>>


Mime
View raw message