falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Seetharam Venkatesh <venkat...@innerzeal.com>
Subject Re: Admin Dashboard
Date Wed, 16 Jul 2014 18:20:23 GMT
Hi Ajay,

Thanks for following this up. Appreciate if you could start a POC
(proof-of-concept) with your ideas and enhance that over time. PLMK if you
need any help in understanding the APIs in falcon. Feel free to create
issues in Jira if you find an issue.

Thanks and looking forward for your contributions.


On Wed, Jul 16, 2014 at 12:13 PM, Ajay Yadav <ajaynsit@gmail.com> wrote:

> Hi ,
>
> Thanks Venkatesh for such an encouraging response. It's absolutely
> wonderful to see a community where new comers are provided such deep
> support for contributing to the project.
>
> However, I am sorry that the discussion has digressed to python vs. java. I
> should have been clearer with the way I phrased it. Let me try to rephrase
> it
>
> *Dust.js alone will not be flexible enough(and insufficient in some cases)
> to build future UI requirements for Admin/Falcon pipeline designer. We need
>  one more server side web framework (which will be delegating core falcon
> tasks to falcon through REST API)*
>
> Again, *I am not advocating/discouraging any client side framework like
> dust.js/bootstrap, just emphasising need of a new component.*
>
> If I try to explain it with an analogy, as per my modest knowledge Oozie
> also exposes REST api but also has a server side webapp.
>
> I interpreted Venkatesh's original question as "we are already using
> dust.js and bootstrap and why do we need more components in the tech
> stack". I tried explaining the need/advantages for a new middle level
> component - a server side framework sitting between clients and REST APIs
> vs. using a client side framework like dust.js on top of REST Api. If you
> read my original email with this context, it will help make things clearer.
>
> Again to help clear Haohui's question, core tasks will still be delegated
> to REST layer which already exists and will be the only place for such
> tasks. However, tasks like user sessions etc. will be done and managed by
> the new component. This layer will manage tasks only specific to Web UI
> like user sessions etc.
>
> With new and ever evolving scenarios like pipeline designer, It will be
> lean since it will be delegating most of tasks to the web server itself but
> at the same time it gives immense flexibility to cater any possible use
> case/extension through plugins in future.
>
>
> *P.S.* Using Django won't cause change in architecture of current
> deployment. It will be separate deployment. Django comes with a built in
> server to aid development and though it is not recommended for production
> set up, it works quite well for 50-80 users. For production level set up,
> It is very flexible in terms of deployment and you can use nginx, gunicorn,
> apache etc.
>
>
>
>
>
> On Tue, Jul 15, 2014 at 5:58 PM, Srikanth Sundarrajan <sriksun@hotmail.com
> >
> wrote:
>
> > Perhaps we can settle on JS + any standard MVC framework in the backend
> if
> > deemed necessary.
> >
> > Regards
> > Srikanth Sundarrajan
> >
> > ----------------------------------------
> > > Date: Tue, 15 Jul 2014 17:25:30 +0530
> > > Subject: Re: Admin Dashboard
> > > From: venkatesh@innerzeal.com
> > > To: dev@falcon.incubator.apache.org
> > >
> > > Hi Ajay,
> > >
> > > Thanks for taking the initiative and sorry for my delayed response. I
> > think
> > > I largely concur with Haohui that keeping the stack lean and using JS
> for
> > > the front end and Java for the backend is desirable.
> > >
> > > Let me know what you need from serverside perspective and one of us can
> > > jump in and help. Since the requirements are fairly trivial and reflect
> > the
> > > state in falcon, we should be able to achieve a lean implementation.
> > >
> > > Today Falcon does not need to run python and embeds a jetty container.
> We
> > > may need to change the deployment architecture to accommodate python.
> > >
> > > Lets take a stab at using any front end tech that you are aware of and
> > > proceed. I'm not hung up on dust nor bootstrap.
> > >
> > > Thanks!
> > >
> > >
> > > On Tue, Jul 15, 2014 at 3:32 AM, Haohui Mai <hmai@hortonworks.com>
> > wrote:
> > >
> > >> bq. Try implementing a
> > >> very basic requirement: user sessions( a logged in user doesn't need
> to
> > >> login again on every page) with existing stack and you will understand
> > what
> > >> I am trying to say.
> > >>
> > >> Note that even simple features like user sessions in Falcon will be
> > coupled
> > >> with other components in the Hadoop ecosystems. In secure set ups,
> > Falcon
> > >> will need to get delegation tokens from the HDFS NameNode and perform
> > >> UGI-related operations. I don't see how Django will help in this case.
> > >>
> > >> bq. P.S. On a side note, you can use python to interact with Hadoop,
> > Hive,
> > >> Pig,
> > >> Oozie. Again, I am not suggesting to use python instead of java in
> > existing
> > >> components.
> > >>
> > >> If this is the case, can you elaborate on how Python+Django can
> benefit
> > >> Falcon, and what exact requirements need to be solved?
> > >>
> > >>
> > >>
> > >>
> > >> On Sun, Jul 13, 2014 at 9:26 PM, Ajay Yadav <ajaynsit@gmail.com>
> wrote:
> > >>
> > >>> Hi Haohui,
> > >>>
> > >>> Thanks for the reply. I am not suggesting to change existing java
> code
> > to
> > >>> python or saying that we can live without javascript. All I am
> > suggesting
> > >>> is that we do need to use a server side web framework. Try
> > implementing a
> > >>> very basic requirement: user sessions( a logged in user doesn't need
> to
> > >>> login again on every page) with existing stack and you will
> understand
> > >> what
> > >>> I am trying to say.
> > >>>
> > >>>
> > >>> P.S. On a side note, you can use python to interact with Hadoop,
> Hive,
> > >> Pig,
> > >>> Oozie. Again, I am not suggesting to use python instead of java in
> > >> existing
> > >>> components.
> > >>>
> > >>>
> > >>> On Sun, Jul 13, 2014 at 11:40 PM, Haohui Mai <hmai@hortonworks.com>
> > >> wrote:
> > >>>
> > >>>> Thanks for your interests in contributing in Falcon. I am one of
the
> > >>>> early contributors in Falcon UI, here are my two cents:
> > >>>>
> > >>>> Falcon has a browser-server architecture which needs to address
two
> > >>>> requirements:
> > >>>>
> > >>>> 1. Provide a feature-rich UI (e.g. FALCON-290) to the browser.
> > >>>>
> > >>>> 2. The server-side logics have to integrate with existing Hadoop
> > >>>> ecosystems (e.g., MapReduce, Hive, Pig, and Oozie) which are heavily
> > >>>> based on Java.
> > >>>>
> > >>>> While personally I'm pretty open to any other technologies, but
I am
> > >>>> yet to be convinced that how Python+Django will add any values
to
> > >>>> address the above requirements. More concretely:
> > >>>>
> > >>>> i. JavaScript is (and will continue to be) an essential part of
the
> UI
> > >>>> to meet requirement (1). The python stack cannot replace the
> > >>>> JavaScript part on the client side.
> > >>>>
> > >>>> ii. Falcon has to rely on Java to interact with other projects
in
> the
> > >>>> Hadoop ecosystems, as there are few bindings in other languages
for
> > >>>> these projects.
> > >>>>
> > >>>>
> > >>>> Using Python+Django might help attract attentions from the python
> > >>>> community, but I think that from the Falcon points of view, it
is
> much
> > >>>> more important to keep align with the whole Hadoop ecosystems (which
> > >> are
> > >>>> the main audiences of the Falcon project). Moreover, I don't think
> > that
> > >>> the
> > >>>> merits that you mentioned for server-side technologies have to
be
> > >> coupled
> > >>>> with Python+Django -- in fact, there are many more implementation
of
> > >>>> enterprise
> > >>>> features like exporting to Excel in Java.
> > >>>>
> > >>>> In short, I think that the current choice of Java+REST+JavaScript
is
> > >>> quite
> > >>>> reasonable. Though I'm pretty open to any other technologies, I'm
> yet
> > >> to
> > >>> be
> > >>>> convinced that Python+Django can be a superior choice.
> > >>>>
> > >>>>
> > >>>> Thanks,
> > >>>> Haohui
> > >>>>
> > >>>>
> > >>>> On Sat, Jul 12, 2014 at 7:06 PM, Ajay Yadav <ajaynsit@gmail.com>
> > >> wrote:
> > >>>>
> > >>>>> Any comments? I am awaiting your decision to start working
on this.
> > >>>>>
> > >>>>>
> > >>>>> On Fri, Jul 11, 2014 at 11:10 AM, Ajay Yadav <ajaynsit@gmail.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> Hi Venkatesh,
> > >>>>>>
> > >>>>>> Valid question and I am glad that you gave an opportunity
to
> > >> discuss
> > >>>> it.
> > >>>>> I
> > >>>>>> have no particular bias towards using django. I have worked
with
> > >>> client
> > >>>>>> side templates(not dust.js but more popular alternatives
like
> > >>> mustache,
> > >>>>>> handlebars etc.) and client side MV* frameworks like backbone.js,
> > >>>>> angularjs
> > >>>>>> etc. It is completely possible to use both together, most
common
> > >>> case.
> > >>>>>>
> > >>>>>> *Note: *I looked at the existing code and saw that the
current
> work
> > >>> in
> > >>>>>> dust.js is very less - around15-25 lines(only dust templates
part
> > >> not
> > >>>>>> html/css). So may I take the liberty of rephrasing your
question
> > >>>> slightly
> > >>>>>> as below
> > >>>>>>
> > >>>>>> *Question*: Assuming we can choose only one to keep stack
mix to
> > >>>> minimum
> > >>>>>> - shall we use pure client side framework(like dust.js)
or pure
> > >>> server
> > >>>>>> side framework(like django)?
> > >>>>>>
> > >>>>>> *Answer*:
> > >>>>>> *TLDR Version:*
> > >>>>>> Server side components offer more flexibility and extensibility
> > >> than
> > >>> a
> > >>>>>> pure client side framework. Using only client side stack
will rule
> > >>> out
> > >>>>> use
> > >>>>>> of database for dashboards, talking to other services,plugins
etc.
> > >> by
> > >>>>> using
> > >>>>>> just dust.js. Things like hardcoding of rest server end
points,
> > >>>> graceful
> > >>>>>> degradation, number of parallel connections available in
browser
> > >> will
> > >>>>> limit
> > >>>>>> pure client side stacks. In other cases it will put a stress
on
> > >>> falcon
> > >>>>> REST
> > >>>>>> API to be modified just for the dashboard. Using python
stack
> > >> instead
> > >>>> of
> > >>>>>> javascript, will help in getting more support from community.
> > >>>>>>
> > >>>>>> *Long Version*
> > >>>>>>
> > >>>>>> - *Convenience vs. Possibility* - Server side components
give us
> > >>>>>> flexibility and extensibility to build for any use case
in
> > >> future
> > >>> as
> > >>>>> well.
> > >>>>>> Any future feature is possible to be built without dust.js
and
> > >>>>> bootstrap
> > >>>>>> but not other way around. Dust.js and bootstrap just make
it
> > >>> easier
> > >>>>> to do
> > >>>>>> lot of things to enhance user experience.
> > >>>>>> - *Features* - Consider the login page of the dashboard.
We will
> > >>>> need
> > >>>>>> a database to store users/ integrate with ldap. We will
need to
> > >>>>> generate
> > >>>>>> new random passwords when users forget passwords. Same
for
> > >>> features
> > >>>>> like
> > >>>>>> bookmarks, custom dashboards etc. A lot of features like
this
> > >>> can't
> > >>>>> be done
> > >>>>>> if we are not using any server side component.
> > >>>>>> - *Sanity of REST API* - REST services of falcon are not
only
> > >> for
> > >>>> web
> > >>>>>> browser consumption. They can be used by command line clients
> > >> etc.
> > >>>> as
> > >>>>> well.
> > >>>>>> As the use cases for dashboard will evolve, it will become
more
> > >>>>> involved to
> > >>>>>> do things using js alone on client side and it will put
lot of
> > >>>> stress
> > >>>>> on
> > >>>>>> modifying REST API to suit just the client. For example
consider
> > >>> the
> > >>>>> use
> > >>>>>> case of allowing an administrator to edit a process from
an HTML
> > >>>>> form. REST
> > >>>>>> API will return plain text/xml and it will be tedious and
slow
> > >> to
> > >>>>> parse
> > >>>>>> everything in javascript side and then recreate it. Sending
a
> > >> json
> > >>>> and
> > >>>>>> accepting json just to cater to web client will be problem
for
> > >>>>>> - Another use case can be to be able to export the reports
like
> > >>> SLAs
> > >>>>>> missed in last quarter, metrics in pdf/excel format and
it will
> > >>> not
> > >>>> be
> > >>>>>> possible to do it on client side and in lack of a server
side
> > >>>>> component it
> > >>>>>> will again put stress on REST API to be modified just for
> > >>> dashboard
> > >>>>> UI.
> > >>>>>> - *Best Practices* - With just dust.js and simple htmls
being
> > >>> served
> > >>>>>> through a webserver, we will have to deal with naming urls
with
> > >>>> .html
> > >>>>>> extensions, hardcoding the REST end points in each HTML
etc.
> > >>>>>> - *Browser constraints* - We will have to deal with browser
> > >>>>>> constraints like number of parallel connections, cross
site
> > >>>> requests,
> > >>>>>> javascript not being enabled etc. This will also restrict
the
> > >>>>> development.
> > >>>>>>
> > >>>>>>
> > >>>>>> - *Community Support* - This is purely my impression and
I might
> > >>> be
> > >>>>>> completely wrong here. I see that more python developers
are
> > >>>> involved
> > >>>>> with
> > >>>>>> big data/data science/machine learning eco system than
more
> > >>>> javascript
> > >>>>>> developers. Hence it will be easier to find more support
for
> > >>>>> maintaining
> > >>>>>> and extending these components in community if we develop
them
> > >> in
> > >>>>> django
> > >>>>>> instead of dust.js.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Fri, Jul 11, 2014 at 2:28 AM, Seetharam Venkatesh <
> > >>>>>> venkatesh@innerzeal.com> wrote:
> > >>>>>>
> > >>>>>>> I think we are using bootstrap and dust templates.
I'd want to
> see
> > >>> if
> > >>>> we
> > >>>>>>> need to introduce more technologies into the mix. Do
we need
> > >> Python
> > >>>> and
> > >>>>>>> Django?
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Thu, Jul 10, 2014 at 6:08 PM, Ajay Yadav <ajaynsit@gmail.com>
> > >>>> wrote:
> > >>>>>>>
> > >>>>>>>> Thanks Srikanth and Venkatesh.
> > >>>>>>>>
> > >>>>>>>> I had looked at the UI and the graph database related
code and I
> > >>>> will
> > >>>>>>> reuse
> > >>>>>>>> existing code where ever available. I checked for
compatibility
> > >> of
> > >>>>> each
> > >>>>>>> of
> > >>>>>>>> the software being used and they are all compatible
with APL 2.0
> > >>>>>>> (Python,
> > >>>>>>>> Django & Bootstrap).
> > >>>>>>>>
> > >>>>>>>> If you can assign the task to me on JIRA also,
it will be
> > >>>>>>>> great(user:ajayyadava) Also, I got the balsamiq
account for
> > >>>>>>> falcon(thanks
> > >>>>>>>> Srikant) and I will be asking all my questions
on mocks there.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Thu, Jul 10, 2014 at 2:45 PM, Seetharam Venkatesh
<
> > >>>>>>>> venkatesh@innerzeal.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> Welcome aboard Ajay. Please let us know how
we can help you to
> > >>> get
> > >>>>> up
> > >>>>>>> and
> > >>>>>>>>> running on this. The REST APIs are documented
at:
> > >>>>>>>>>
> > >>> http://falcon.incubator.apache.org/docs/restapi/ResourceList.html
> > >>>>>>>>>
> > >>>>>>>>> Thanks!
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Thu, Jul 10, 2014 at 7:12 AM, Ajay Yadav
<
> > >> ajaynsit@gmail.com
> > >>>>
> > >>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi,
> > >>>>>>>>>>
> > >>>>>>>>>> I would like to volunteer for
> > >>>>>>>>>> https://issues.apache.org/jira/browse/FALCON-37
> > >>>>>>>>>>
> > >>>>>>>>>> I have more than 7 years of experience
in building
> > >> websites. I
> > >>>>> have
> > >>>>>>>>>> experience in both client side
> > >>>>>>> technologies(javascript/HTML5/Bootstrap
> > >>>>>>>>>> etc.) and server side technologies(Python/Django).
> > >>>>>>>>>>
> > >>>>>>>>>> I will need some help with iterating on
the mocks as they
> > >>> don't
> > >>>>>>> seem to
> > >>>>>>>>> be
> > >>>>>>>>>> complete. I spent couple of hours on building
a couple of
> > >>> pages
> > >>>>>>> using
> > >>>>>>>>>> "dummy" data to get an idea of what all
APIs are required.
> > >>>> Please
> > >>>>>>> find
> > >>>>>>>>> them
> > >>>>>>>>>> attached. They are just a quick and dirty
prototype and UI
> > >> is
> > >>>>> pretty
> > >>>>>>>> raw
> > >>>>>>>>> as
> > >>>>>>>>>> of now, to get quick feedback. I used Django,HTML5
&
> > >>> Bootstrap.
> > >>>>>>>>>>
> > >>>>>>>>>> Please let me know if I can own this task.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Regards,
> > >>>>>>>>> Venkatesh
> > >>>>>>>>>
> > >>>>>>>>> “Perfection (in design) is achieved not when
there is nothing
> > >>> more
> > >>>>> to
> > >>>>>>>> add,
> > >>>>>>>>> but rather when there is nothing more to take
away.”
> > >>>>>>>>> - Antoine de Saint-Exupéry
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Regards,
> > >>>>>>> Venkatesh
> > >>>>>>>
> > >>>>>>> “Perfection (in design) is achieved not when there
is nothing
> more
> > >>> to
> > >>>>> add,
> > >>>>>>> but rather when there is nothing more to take away.”
> > >>>>>>> - Antoine de Saint-Exupéry
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>> --
> > >>>> CONFIDENTIALITY NOTICE
> > >>>> NOTICE: This message is intended for the use of the individual
or
> > >> entity
> > >>> to
> > >>>> which it is addressed and may contain information that is
> > confidential,
> > >>>> privileged and exempt from disclosure under applicable law. If
the
> > >> reader
> > >>>> of this message is not the intended recipient, you are hereby
> notified
> > >>> that
> > >>>> any printing, copying, dissemination, distribution, disclosure
or
> > >>>> forwarding of this communication is strictly prohibited. If you
have
> > >>>> received this communication in error, please contact the sender
> > >>> immediately
> > >>>> and delete it from your system. Thank You.
> > >>>>
> > >>>
> > >>
> > >> --
> > >> CONFIDENTIALITY NOTICE
> > >> NOTICE: This message is intended for the use of the individual or
> > entity to
> > >> which it is addressed and may contain information that is
> confidential,
> > >> privileged and exempt from disclosure under applicable law. If the
> > reader
> > >> of this message is not the intended recipient, you are hereby notified
> > that
> > >> any printing, copying, dissemination, distribution, disclosure or
> > >> forwarding of this communication is strictly prohibited. If you have
> > >> received this communication in error, please contact the sender
> > immediately
> > >> and delete it from your system. Thank You.
> > >>
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Venkatesh
> > >
> > > “Perfection (in design) is achieved not when there is nothing more to
> > add,
> > > but rather when there is nothing more to take away.”
> > > - Antoine de Saint-Exupéry
> >
> >
>



-- 
Regards,
Venkatesh

“Perfection (in design) is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message