falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ajay Yadav <ajayn...@gmail.com>
Subject Re: Admin Dashboard
Date Wed, 16 Jul 2014 06:43:44 GMT
Hi ,

Thanks Venkatesh for such an encouraging response. It's absolutely
wonderful to see a community where new comers are provided such deep
support for contributing to the project.

However, I am sorry that the discussion has digressed to python vs. java. I
should have been clearer with the way I phrased it. Let me try to rephrase
it

*Dust.js alone will not be flexible enough(and insufficient in some cases)
to build future UI requirements for Admin/Falcon pipeline designer. We need
 one more server side web framework (which will be delegating core falcon
tasks to falcon through REST API)*

Again, *I am not advocating/discouraging any client side framework like
dust.js/bootstrap, just emphasising need of a new component.*

If I try to explain it with an analogy, as per my modest knowledge Oozie
also exposes REST api but also has a server side webapp.

I interpreted Venkatesh's original question as "we are already using
dust.js and bootstrap and why do we need more components in the tech
stack". I tried explaining the need/advantages for a new middle level
component - a server side framework sitting between clients and REST APIs
vs. using a client side framework like dust.js on top of REST Api. If you
read my original email with this context, it will help make things clearer.

Again to help clear Haohui's question, core tasks will still be delegated
to REST layer which already exists and will be the only place for such
tasks. However, tasks like user sessions etc. will be done and managed by
the new component. This layer will manage tasks only specific to Web UI
like user sessions etc.

With new and ever evolving scenarios like pipeline designer, It will be
lean since it will be delegating most of tasks to the web server itself but
at the same time it gives immense flexibility to cater any possible use
case/extension through plugins in future.


*P.S.* Using Django won't cause change in architecture of current
deployment. It will be separate deployment. Django comes with a built in
server to aid development and though it is not recommended for production
set up, it works quite well for 50-80 users. For production level set up,
It is very flexible in terms of deployment and you can use nginx, gunicorn,
apache etc.





On Tue, Jul 15, 2014 at 5:58 PM, Srikanth Sundarrajan <sriksun@hotmail.com>
wrote:

> Perhaps we can settle on JS + any standard MVC framework in the backend if
> deemed necessary.
>
> Regards
> Srikanth Sundarrajan
>
> ----------------------------------------
> > Date: Tue, 15 Jul 2014 17:25:30 +0530
> > Subject: Re: Admin Dashboard
> > From: venkatesh@innerzeal.com
> > To: dev@falcon.incubator.apache.org
> >
> > Hi Ajay,
> >
> > Thanks for taking the initiative and sorry for my delayed response. I
> think
> > I largely concur with Haohui that keeping the stack lean and using JS for
> > the front end and Java for the backend is desirable.
> >
> > Let me know what you need from serverside perspective and one of us can
> > jump in and help. Since the requirements are fairly trivial and reflect
> the
> > state in falcon, we should be able to achieve a lean implementation.
> >
> > Today Falcon does not need to run python and embeds a jetty container. We
> > may need to change the deployment architecture to accommodate python.
> >
> > Lets take a stab at using any front end tech that you are aware of and
> > proceed. I'm not hung up on dust nor bootstrap.
> >
> > Thanks!
> >
> >
> > On Tue, Jul 15, 2014 at 3:32 AM, Haohui Mai <hmai@hortonworks.com>
> wrote:
> >
> >> bq. Try implementing a
> >> very basic requirement: user sessions( a logged in user doesn't need to
> >> login again on every page) with existing stack and you will understand
> what
> >> I am trying to say.
> >>
> >> Note that even simple features like user sessions in Falcon will be
> coupled
> >> with other components in the Hadoop ecosystems. In secure set ups,
> Falcon
> >> will need to get delegation tokens from the HDFS NameNode and perform
> >> UGI-related operations. I don't see how Django will help in this case.
> >>
> >> bq. P.S. On a side note, you can use python to interact with Hadoop,
> Hive,
> >> Pig,
> >> Oozie. Again, I am not suggesting to use python instead of java in
> existing
> >> components.
> >>
> >> If this is the case, can you elaborate on how Python+Django can benefit
> >> Falcon, and what exact requirements need to be solved?
> >>
> >>
> >>
> >>
> >> On Sun, Jul 13, 2014 at 9:26 PM, Ajay Yadav <ajaynsit@gmail.com> wrote:
> >>
> >>> Hi Haohui,
> >>>
> >>> Thanks for the reply. I am not suggesting to change existing java code
> to
> >>> python or saying that we can live without javascript. All I am
> suggesting
> >>> is that we do need to use a server side web framework. Try
> implementing a
> >>> very basic requirement: user sessions( a logged in user doesn't need to
> >>> login again on every page) with existing stack and you will understand
> >> what
> >>> I am trying to say.
> >>>
> >>>
> >>> P.S. On a side note, you can use python to interact with Hadoop, Hive,
> >> Pig,
> >>> Oozie. Again, I am not suggesting to use python instead of java in
> >> existing
> >>> components.
> >>>
> >>>
> >>> On Sun, Jul 13, 2014 at 11:40 PM, Haohui Mai <hmai@hortonworks.com>
> >> wrote:
> >>>
> >>>> Thanks for your interests in contributing in Falcon. I am one of the
> >>>> early contributors in Falcon UI, here are my two cents:
> >>>>
> >>>> Falcon has a browser-server architecture which needs to address two
> >>>> requirements:
> >>>>
> >>>> 1. Provide a feature-rich UI (e.g. FALCON-290) to the browser.
> >>>>
> >>>> 2. The server-side logics have to integrate with existing Hadoop
> >>>> ecosystems (e.g., MapReduce, Hive, Pig, and Oozie) which are heavily
> >>>> based on Java.
> >>>>
> >>>> While personally I'm pretty open to any other technologies, but I am
> >>>> yet to be convinced that how Python+Django will add any values to
> >>>> address the above requirements. More concretely:
> >>>>
> >>>> i. JavaScript is (and will continue to be) an essential part of the
UI
> >>>> to meet requirement (1). The python stack cannot replace the
> >>>> JavaScript part on the client side.
> >>>>
> >>>> ii. Falcon has to rely on Java to interact with other projects in the
> >>>> Hadoop ecosystems, as there are few bindings in other languages for
> >>>> these projects.
> >>>>
> >>>>
> >>>> Using Python+Django might help attract attentions from the python
> >>>> community, but I think that from the Falcon points of view, it is much
> >>>> more important to keep align with the whole Hadoop ecosystems (which
> >> are
> >>>> the main audiences of the Falcon project). Moreover, I don't think
> that
> >>> the
> >>>> merits that you mentioned for server-side technologies have to be
> >> coupled
> >>>> with Python+Django -- in fact, there are many more implementation of
> >>>> enterprise
> >>>> features like exporting to Excel in Java.
> >>>>
> >>>> In short, I think that the current choice of Java+REST+JavaScript is
> >>> quite
> >>>> reasonable. Though I'm pretty open to any other technologies, I'm yet
> >> to
> >>> be
> >>>> convinced that Python+Django can be a superior choice.
> >>>>
> >>>>
> >>>> Thanks,
> >>>> Haohui
> >>>>
> >>>>
> >>>> On Sat, Jul 12, 2014 at 7:06 PM, Ajay Yadav <ajaynsit@gmail.com>
> >> wrote:
> >>>>
> >>>>> Any comments? I am awaiting your decision to start working on this.
> >>>>>
> >>>>>
> >>>>> On Fri, Jul 11, 2014 at 11:10 AM, Ajay Yadav <ajaynsit@gmail.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi Venkatesh,
> >>>>>>
> >>>>>> Valid question and I am glad that you gave an opportunity to
> >> discuss
> >>>> it.
> >>>>> I
> >>>>>> have no particular bias towards using django. I have worked
with
> >>> client
> >>>>>> side templates(not dust.js but more popular alternatives like
> >>> mustache,
> >>>>>> handlebars etc.) and client side MV* frameworks like backbone.js,
> >>>>> angularjs
> >>>>>> etc. It is completely possible to use both together, most common
> >>> case.
> >>>>>>
> >>>>>> *Note: *I looked at the existing code and saw that the current
work
> >>> in
> >>>>>> dust.js is very less - around15-25 lines(only dust templates
part
> >> not
> >>>>>> html/css). So may I take the liberty of rephrasing your question
> >>>> slightly
> >>>>>> as below
> >>>>>>
> >>>>>> *Question*: Assuming we can choose only one to keep stack mix
to
> >>>> minimum
> >>>>>> - shall we use pure client side framework(like dust.js) or pure
> >>> server
> >>>>>> side framework(like django)?
> >>>>>>
> >>>>>> *Answer*:
> >>>>>> *TLDR Version:*
> >>>>>> Server side components offer more flexibility and extensibility
> >> than
> >>> a
> >>>>>> pure client side framework. Using only client side stack will
rule
> >>> out
> >>>>> use
> >>>>>> of database for dashboards, talking to other services,plugins
etc.
> >> by
> >>>>> using
> >>>>>> just dust.js. Things like hardcoding of rest server end points,
> >>>> graceful
> >>>>>> degradation, number of parallel connections available in browser
> >> will
> >>>>> limit
> >>>>>> pure client side stacks. In other cases it will put a stress
on
> >>> falcon
> >>>>> REST
> >>>>>> API to be modified just for the dashboard. Using python stack
> >> instead
> >>>> of
> >>>>>> javascript, will help in getting more support from community.
> >>>>>>
> >>>>>> *Long Version*
> >>>>>>
> >>>>>> - *Convenience vs. Possibility* - Server side components give
us
> >>>>>> flexibility and extensibility to build for any use case in
> >> future
> >>> as
> >>>>> well.
> >>>>>> Any future feature is possible to be built without dust.js and
> >>>>> bootstrap
> >>>>>> but not other way around. Dust.js and bootstrap just make it
> >>> easier
> >>>>> to do
> >>>>>> lot of things to enhance user experience.
> >>>>>> - *Features* - Consider the login page of the dashboard. We
will
> >>>> need
> >>>>>> a database to store users/ integrate with ldap. We will need
to
> >>>>> generate
> >>>>>> new random passwords when users forget passwords. Same for
> >>> features
> >>>>> like
> >>>>>> bookmarks, custom dashboards etc. A lot of features like this
> >>> can't
> >>>>> be done
> >>>>>> if we are not using any server side component.
> >>>>>> - *Sanity of REST API* - REST services of falcon are not only
> >> for
> >>>> web
> >>>>>> browser consumption. They can be used by command line clients
> >> etc.
> >>>> as
> >>>>> well.
> >>>>>> As the use cases for dashboard will evolve, it will become more
> >>>>> involved to
> >>>>>> do things using js alone on client side and it will put lot
of
> >>>> stress
> >>>>> on
> >>>>>> modifying REST API to suit just the client. For example consider
> >>> the
> >>>>> use
> >>>>>> case of allowing an administrator to edit a process from an
HTML
> >>>>> form. REST
> >>>>>> API will return plain text/xml and it will be tedious and slow
> >> to
> >>>>> parse
> >>>>>> everything in javascript side and then recreate it. Sending
a
> >> json
> >>>> and
> >>>>>> accepting json just to cater to web client will be problem for
> >>>>>> - Another use case can be to be able to export the reports like
> >>> SLAs
> >>>>>> missed in last quarter, metrics in pdf/excel format and it will
> >>> not
> >>>> be
> >>>>>> possible to do it on client side and in lack of a server side
> >>>>> component it
> >>>>>> will again put stress on REST API to be modified just for
> >>> dashboard
> >>>>> UI.
> >>>>>> - *Best Practices* - With just dust.js and simple htmls being
> >>> served
> >>>>>> through a webserver, we will have to deal with naming urls with
> >>>> .html
> >>>>>> extensions, hardcoding the REST end points in each HTML etc.
> >>>>>> - *Browser constraints* - We will have to deal with browser
> >>>>>> constraints like number of parallel connections, cross site
> >>>> requests,
> >>>>>> javascript not being enabled etc. This will also restrict the
> >>>>> development.
> >>>>>>
> >>>>>>
> >>>>>> - *Community Support* - This is purely my impression and I might
> >>> be
> >>>>>> completely wrong here. I see that more python developers are
> >>>> involved
> >>>>> with
> >>>>>> big data/data science/machine learning eco system than more
> >>>> javascript
> >>>>>> developers. Hence it will be easier to find more support for
> >>>>> maintaining
> >>>>>> and extending these components in community if we develop them
> >> in
> >>>>> django
> >>>>>> instead of dust.js.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Jul 11, 2014 at 2:28 AM, Seetharam Venkatesh <
> >>>>>> venkatesh@innerzeal.com> wrote:
> >>>>>>
> >>>>>>> I think we are using bootstrap and dust templates. I'd want
to see
> >>> if
> >>>> we
> >>>>>>> need to introduce more technologies into the mix. Do we
need
> >> Python
> >>>> and
> >>>>>>> Django?
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Jul 10, 2014 at 6:08 PM, Ajay Yadav <ajaynsit@gmail.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Thanks Srikanth and Venkatesh.
> >>>>>>>>
> >>>>>>>> I had looked at the UI and the graph database related
code and I
> >>>> will
> >>>>>>> reuse
> >>>>>>>> existing code where ever available. I checked for compatibility
> >> of
> >>>>> each
> >>>>>>> of
> >>>>>>>> the software being used and they are all compatible
with APL 2.0
> >>>>>>> (Python,
> >>>>>>>> Django & Bootstrap).
> >>>>>>>>
> >>>>>>>> If you can assign the task to me on JIRA also, it will
be
> >>>>>>>> great(user:ajayyadava) Also, I got the balsamiq account
for
> >>>>>>> falcon(thanks
> >>>>>>>> Srikant) and I will be asking all my questions on mocks
there.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Jul 10, 2014 at 2:45 PM, Seetharam Venkatesh
<
> >>>>>>>> venkatesh@innerzeal.com> wrote:
> >>>>>>>>
> >>>>>>>>> Welcome aboard Ajay. Please let us know how we can
help you to
> >>> get
> >>>>> up
> >>>>>>> and
> >>>>>>>>> running on this. The REST APIs are documented at:
> >>>>>>>>>
> >>> http://falcon.incubator.apache.org/docs/restapi/ResourceList.html
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Thu, Jul 10, 2014 at 7:12 AM, Ajay Yadav <
> >> ajaynsit@gmail.com
> >>>>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi,
> >>>>>>>>>>
> >>>>>>>>>> I would like to volunteer for
> >>>>>>>>>> https://issues.apache.org/jira/browse/FALCON-37
> >>>>>>>>>>
> >>>>>>>>>> I have more than 7 years of experience in building
> >> websites. I
> >>>>> have
> >>>>>>>>>> experience in both client side
> >>>>>>> technologies(javascript/HTML5/Bootstrap
> >>>>>>>>>> etc.) and server side technologies(Python/Django).
> >>>>>>>>>>
> >>>>>>>>>> I will need some help with iterating on the
mocks as they
> >>> don't
> >>>>>>> seem to
> >>>>>>>>> be
> >>>>>>>>>> complete. I spent couple of hours on building
a couple of
> >>> pages
> >>>>>>> using
> >>>>>>>>>> "dummy" data to get an idea of what all APIs
are required.
> >>>> Please
> >>>>>>> find
> >>>>>>>>> them
> >>>>>>>>>> attached. They are just a quick and dirty prototype
and UI
> >> is
> >>>>> pretty
> >>>>>>>> raw
> >>>>>>>>> as
> >>>>>>>>>> of now, to get quick feedback. I used Django,HTML5
&
> >>> Bootstrap.
> >>>>>>>>>>
> >>>>>>>>>> Please let me know if I can own this task.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Regards,
> >>>>>>>>> Venkatesh
> >>>>>>>>>
> >>>>>>>>> “Perfection (in design) is achieved not when there
is nothing
> >>> more
> >>>>> to
> >>>>>>>> add,
> >>>>>>>>> but rather when there is nothing more to take away.”
> >>>>>>>>> - Antoine de Saint-Exupéry
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Regards,
> >>>>>>> Venkatesh
> >>>>>>>
> >>>>>>> “Perfection (in design) is achieved not when there is
nothing more
> >>> to
> >>>>> add,
> >>>>>>> but rather when there is nothing more to take away.”
> >>>>>>> - Antoine de Saint-Exupéry
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>> --
> >>>> CONFIDENTIALITY NOTICE
> >>>> NOTICE: This message is intended for the use of the individual or
> >> entity
> >>> to
> >>>> which it is addressed and may contain information that is
> confidential,
> >>>> privileged and exempt from disclosure under applicable law. If the
> >> reader
> >>>> of this message is not the intended recipient, you are hereby notified
> >>> that
> >>>> any printing, copying, dissemination, distribution, disclosure or
> >>>> forwarding of this communication is strictly prohibited. If you have
> >>>> received this communication in error, please contact the sender
> >>> immediately
> >>>> and delete it from your system. Thank You.
> >>>>
> >>>
> >>
> >> --
> >> CONFIDENTIALITY NOTICE
> >> NOTICE: This message is intended for the use of the individual or
> entity to
> >> which it is addressed and may contain information that is confidential,
> >> privileged and exempt from disclosure under applicable law. If the
> reader
> >> of this message is not the intended recipient, you are hereby notified
> that
> >> any printing, copying, dissemination, distribution, disclosure or
> >> forwarding of this communication is strictly prohibited. If you have
> >> received this communication in error, please contact the sender
> immediately
> >> and delete it from your system. Thank You.
> >>
> >
> >
> >
> > --
> > Regards,
> > Venkatesh
> >
> > “Perfection (in design) is achieved not when there is nothing more to
> add,
> > but rather when there is nothing more to take away.”
> > - Antoine de Saint-Exupéry
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message