Mailing-List: contact dev-help@falcon.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@falcon.apache.org
Received-SPF: pass (nike.apache.org: domain of arpit@hortonworks.com
 designates 64.78.52.184 as permitted sender)
Subject: Re: [DISCUSS]: Moving recipe cooking to server
MIME-Version: 1.0
From: Arpit Gupta <arpit@hortonworks.com>
To: "dev@falcon.apache.org" <dev@falcon.apache.org>
Thread-Topic: [DISCUSS]: Moving recipe cooking to server
Thread-Index: 
 AQHQXIPOQ92fT7FURE2EE8sWKb8V+50Y3kKA//+ZJACAAHfVAP//+nadgAB2EAD//5Vy1QAPNCOA
Date: Thu, 12 Mar 2015 15:28:57 +0000
Message-ID: <BBEC0020-BF9E-4870-859E-1D86F73836ED@hortonworks.com>
References: <D1266F6B.2D69D%sramesh@hortonworks.com>
 <BLU179-W65B2987383839049F55E10A4060@phx.gbl>
 <D1268668.1D91%yzheng@hortonworks.com>
 <BLU179-W324C0DCEB3ADCDB11FFF3BA4060@phx.gbl>
 <65503bb5eb734be9bcb668a0bc1809d7@MBX080-W3-CO-1.exch080.serverpod.net>
 <,<BLU406-EAS18923A4E83D5F103F2E44FDA4060@phx.gbl> <>>
 <1799159e3b8e4ec0a0add04d0bc33974@MBX080-W3-CO-1.exch080.serverpod.net>
In-Reply-To: 
 <1799159e3b8e4ec0a0add04d0bc33974@MBX080-W3-CO-1.exch080.serverpod.net>
Accept-Language: en-US
Content-Language: en-US
Content-Type: multipart/alternative;
	boundary="_000_BBEC0020BF9E4870859E1D86F73836EDhortonworkscom_"

--_000_BBEC0020BF9E4870859E1D86F73836EDhortonworkscom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Are people not moving away from JSP's? I know all UI's in Hadoop moved away=
 from JSP recently to html 5 https://issues.apache.org/jira/browse/HADOOP-1=
0563

--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/

On Mar 12, 2015, at 8:13 AM, Ying Zheng <yzheng@hortonworks.com<mailto:yzhe=
ng@hortonworks.com>> wrote:

Sort of, but not a completely new server. We don't need to write a new serv=
er. As long as we have Tomcat installed and running, we just need to copy f=
ront end JSP files to a folder under tomcat. In this way, we separate front=
 end and back end.

On Mar 12, 2015 7:39 AM, Srikanth Sundarrajan <sriksun@hotmail.com<mailto:s=
riksun@hotmail.com>> wrote:
Are you suggesting a completely new server for cooking recipe?

Sent from my iPhone

On 12-Mar-2015, at 8:03 pm, "Ying Zheng" <yzheng@hortonworks.com<mailto:yzh=
eng@hortonworks.com>> wrote:

Perhaps I don't understand your question. JSP is one way to generate webpag=
es. Why does it need to run in Falcon server? It can be isolated from Falco=
n server, and developed and run in Apache Tomcat server.

Thanks,
Ying

On Mar 12, 2015 12:53 AM, Srikanth Sundarrajan <sriksun@hotmail.com<mailto:=
sriksun@hotmail.com>> wrote:
Hi Ying,
  JSP would still run in Falcon server JVM, so not sure if that is any diff=
erent from implementing this natively inside the falcon server.

Regards
Srikanth Sundarrajan

----------------------------------------
Subject: Re: [DISCUSS]: Moving recipe cooking to server
From: yzheng@hortonworks.com<mailto:yzheng@hortonworks.com>
To: dev@falcon.apache.org<mailto:dev@falcon.apache.org>
Date: Thu, 12 Mar 2015 07:43:28 +0000

Hi Sowmya, Srikanth, Venkatesh and Venkat,

As far as I understand, recipe helps the user generate the process entity
xml and it looks best to keep it a client side logic, as what you did
before. The current question is how to run recipe logic on DR UI. To avoid
repeating all the recipe logic in JS, one choice you can consider is JSP.
JSP can be supported by Apache Tomcat server. If you package the recipe
code into JAR, it can be reused in JSP. JS and CSS can also be included in
the JSP page.

Cheers,
Ying


On 3/11/15, 11:51 PM, "Srikanth Sundarrajan" <sriksun@hotmail.com<mailto:sr=
iksun@hotmail.com>> wrote:

Thanks Sowmya for capturing the discussion and also initiating this
conversation to drive consensus around the design. I am inclined towards
Approach 2.

Regards
Srikanth Sundarrajan

----------------------------------------
Subject: [DISCUSS]: Moving recipe cooking to server
From: sramesh@hortonworks.com<mailto:sramesh@hortonworks.com>
To: dev@falcon.apache.org<mailto:dev@falcon.apache.org>
Date: Thu, 12 Mar 2015 05:17:16 +0000

I had a discussion with Srikanth, Venkatesh and Venkat regarding making
recipe processing server side concept. I am sending out the summary of
the meeting and opening it for further discussion.

Today Recipe cooking is a client side logic. Recipe also supports
extensions i.e. user can cook his/her own custom recipes.
Decision to make it client side logic was for the following reasons

* Keep it isolated from falcon server

* As custom recipe cooking is supported, user recipes can introduce
security vulnerabilities and also can bring down the falcon server

Today, falcon provides HDFS DR recipe out of the box. There is a plan
to add UI support for DR in Falcon.
Rest API support cannot be added for recipe as it is client side
processing.
If the UI is pure java script[JS] then all the recipe cooking logic has
to be repeated in JS. This is not a feasible solution - if more recipes
are added say DR for hive, hbase and others, UI won't be extensible.

For the above mentioned reasons Recipe should me made a server side
logic.
Provided recipes [recipes provided out of the box] can run as Falcon
process. Recipe cooking will be done in a new process if its custom
recipe [user code].

For cooking of custom recipes, design proposed should consider handling
security implications, handling the issues where the custom user code
can bring down the Falcon server (trapping System.exit), handling class
path isolation.
Also it shouldn't in anyway destabilize the Falcon system.

There are couple of approaches which was discussed

Approach 1:
Custom Recipe cooking can be carried out separately in another Oozie
WF, this will ensure isolation. Oozie already has the ability to
schedule jobs as a user and handles all the security aspects of it.

Pros:
- Provides isolation
- Piggyback on Oozie as it already provides the required functionality

Cons:
- As recipe processing is done in different WF, from operations point
of view user cannot figure out recipe processing status and thus adds to
the operational pain.

Approach 2:
Custom recipe cooking is done on the server side in a separate
independent process than Falcon process I.e. It runs in a different JVM.
Throttling should be added for how many recipe cooking processes can be
launched keeping in mind the machine configuration.

Pros:
- Provides isolation as recipe cooking is done in a independent process

Cons:
- Performance overhead as new process is launched for custom recipe
cooking
- Adds more complexity to the system

Thoughts?

Thanks,
Sowmya


--_000_BBEC0020BF9E4870859E1D86F73836EDhortonworkscom_--