falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srikanth Sundarrajan <srik...@hotmail.com>
Subject RE: [DISCUSS]: Moving recipe cooking to server
Date Thu, 12 Mar 2015 07:52:16 GMT
Hi Ying,
    JSP would still run in Falcon server JVM, so not sure if that is any different from
implementing this natively inside the falcon server.

Srikanth Sundarrajan

> Subject: Re: [DISCUSS]: Moving recipe cooking to server
> From: yzheng@hortonworks.com
> To: dev@falcon.apache.org
> Date: Thu, 12 Mar 2015 07:43:28 +0000
> Hi Sowmya, Srikanth, Venkatesh and Venkat,
> As far as I understand, recipe helps the user generate the process entity
> xml and it looks best to keep it a client side logic, as what you did
> before. The current question is how to run recipe logic on DR UI. To avoid
> repeating all the recipe logic in JS, one choice you can consider is JSP.
> JSP can be supported by Apache Tomcat server. If you package the recipe
> code into JAR, it can be reused in JSP. JS and CSS can also be included in
> the JSP page.
> Cheers,
> Ying
> On 3/11/15, 11:51 PM, "Srikanth Sundarrajan" <sriksun@hotmail.com> wrote:
>>Thanks Sowmya for capturing the discussion and also initiating this
>>conversation to drive consensus around the design. I am inclined towards
>>Approach 2.
>>Srikanth Sundarrajan
>>> Subject: [DISCUSS]: Moving recipe cooking to server
>>> From: sramesh@hortonworks.com
>>> To: dev@falcon.apache.org
>>> Date: Thu, 12 Mar 2015 05:17:16 +0000
>>> I had a discussion with Srikanth, Venkatesh and Venkat regarding making
>>>recipe processing server side concept. I am sending out the summary of
>>>the meeting and opening it for further discussion.
>>> Today Recipe cooking is a client side logic. Recipe also supports
>>>extensions i.e. user can cook his/her own custom recipes.
>>> Decision to make it client side logic was for the following reasons
>>> * Keep it isolated from falcon server
>>> * As custom recipe cooking is supported, user recipes can introduce
>>>security vulnerabilities and also can bring down the falcon server
>>> Today, falcon provides HDFS DR recipe out of the box. There is a plan
>>>to add UI support for DR in Falcon.
>>> Rest API support cannot be added for recipe as it is client side
>>> If the UI is pure java script[JS] then all the recipe cooking logic has
>>>to be repeated in JS. This is not a feasible solution - if more recipes
>>>are added say DR for hive, hbase and others, UI won't be extensible.
>>> For the above mentioned reasons Recipe should me made a server side
>>> Provided recipes [recipes provided out of the box] can run as Falcon
>>>process. Recipe cooking will be done in a new process if its custom
>>>recipe [user code].
>>> For cooking of custom recipes, design proposed should consider handling
>>>security implications, handling the issues where the custom user code
>>>can bring down the Falcon server (trapping System.exit), handling class
>>>path isolation.
>>> Also it shouldn't in anyway destabilize the Falcon system.
>>> There are couple of approaches which was discussed
>>> Approach 1:
>>> Custom Recipe cooking can be carried out separately in another Oozie
>>>WF, this will ensure isolation. Oozie already has the ability to
>>>schedule jobs as a user and handles all the security aspects of it.
>>> Pros:
>>> - Provides isolation
>>> - Piggyback on Oozie as it already provides the required functionality
>>> Cons:
>>> - As recipe processing is done in different WF, from operations point
>>>of view user cannot figure out recipe processing status and thus adds to
>>>the operational pain.
>>> Approach 2:
>>> Custom recipe cooking is done on the server side in a separate
>>>independent process than Falcon process I.e. It runs in a different JVM.
>>>Throttling should be added for how many recipe cooking processes can be
>>>launched keeping in mind the machine configuration.
>>> Pros:
>>> - Provides isolation as recipe cooking is done in a independent process
>>> Cons:
>>> - Performance overhead as new process is launched for custom recipe
>>> - Adds more complexity to the system
>>> Thoughts?
>>> Thanks,
>>> Sowmya
View raw message