falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ajay Yadav <ajayn...@gmail.com>
Subject Re: Admin Dashboard
Date Thu, 17 Jul 2014 04:25:54 GMT
Hi Venkatesh,

Thanks! I will start right away and update once I have a POC ready.


On Wed, Jul 16, 2014 at 11:50 PM, Seetharam Venkatesh <
venkatesh@innerzeal.com> wrote:

> Hi Ajay,
>
> Thanks for following this up. Appreciate if you could start a POC
> (proof-of-concept) with your ideas and enhance that over time. PLMK if you
> need any help in understanding the APIs in falcon. Feel free to create
> issues in Jira if you find an issue.
>
> Thanks and looking forward for your contributions.
>
>
> On Wed, Jul 16, 2014 at 12:13 PM, Ajay Yadav <ajaynsit@gmail.com> wrote:
>
> > Hi ,
> >
> > Thanks Venkatesh for such an encouraging response. It's absolutely
> > wonderful to see a community where new comers are provided such deep
> > support for contributing to the project.
> >
> > However, I am sorry that the discussion has digressed to python vs.
> java. I
> > should have been clearer with the way I phrased it. Let me try to
> rephrase
> > it
> >
> > *Dust.js alone will not be flexible enough(and insufficient in some
> cases)
> > to build future UI requirements for Admin/Falcon pipeline designer. We
> need
> >  one more server side web framework (which will be delegating core falcon
> > tasks to falcon through REST API)*
> >
> > Again, *I am not advocating/discouraging any client side framework like
> > dust.js/bootstrap, just emphasising need of a new component.*
> >
> > If I try to explain it with an analogy, as per my modest knowledge Oozie
> > also exposes REST api but also has a server side webapp.
> >
> > I interpreted Venkatesh's original question as "we are already using
> > dust.js and bootstrap and why do we need more components in the tech
> > stack". I tried explaining the need/advantages for a new middle level
> > component - a server side framework sitting between clients and REST APIs
> > vs. using a client side framework like dust.js on top of REST Api. If you
> > read my original email with this context, it will help make things
> clearer.
> >
> > Again to help clear Haohui's question, core tasks will still be delegated
> > to REST layer which already exists and will be the only place for such
> > tasks. However, tasks like user sessions etc. will be done and managed by
> > the new component. This layer will manage tasks only specific to Web UI
> > like user sessions etc.
> >
> > With new and ever evolving scenarios like pipeline designer, It will be
> > lean since it will be delegating most of tasks to the web server itself
> but
> > at the same time it gives immense flexibility to cater any possible use
> > case/extension through plugins in future.
> >
> >
> > *P.S.* Using Django won't cause change in architecture of current
> > deployment. It will be separate deployment. Django comes with a built in
> > server to aid development and though it is not recommended for production
> > set up, it works quite well for 50-80 users. For production level set up,
> > It is very flexible in terms of deployment and you can use nginx,
> gunicorn,
> > apache etc.
> >
> >
> >
> >
> >
> > On Tue, Jul 15, 2014 at 5:58 PM, Srikanth Sundarrajan <
> sriksun@hotmail.com
> > >
> > wrote:
> >
> > > Perhaps we can settle on JS + any standard MVC framework in the backend
> > if
> > > deemed necessary.
> > >
> > > Regards
> > > Srikanth Sundarrajan
> > >
> > > ----------------------------------------
> > > > Date: Tue, 15 Jul 2014 17:25:30 +0530
> > > > Subject: Re: Admin Dashboard
> > > > From: venkatesh@innerzeal.com
> > > > To: dev@falcon.incubator.apache.org
> > > >
> > > > Hi Ajay,
> > > >
> > > > Thanks for taking the initiative and sorry for my delayed response. I
> > > think
> > > > I largely concur with Haohui that keeping the stack lean and using JS
> > for
> > > > the front end and Java for the backend is desirable.
> > > >
> > > > Let me know what you need from serverside perspective and one of us
> can
> > > > jump in and help. Since the requirements are fairly trivial and
> reflect
> > > the
> > > > state in falcon, we should be able to achieve a lean implementation.
> > > >
> > > > Today Falcon does not need to run python and embeds a jetty
> container.
> > We
> > > > may need to change the deployment architecture to accommodate python.
> > > >
> > > > Lets take a stab at using any front end tech that you are aware of
> and
> > > > proceed. I'm not hung up on dust nor bootstrap.
> > > >
> > > > Thanks!
> > > >
> > > >
> > > > On Tue, Jul 15, 2014 at 3:32 AM, Haohui Mai <hmai@hortonworks.com>
> > > wrote:
> > > >
> > > >> bq. Try implementing a
> > > >> very basic requirement: user sessions( a logged in user doesn't need
> > to
> > > >> login again on every page) with existing stack and you will
> understand
> > > what
> > > >> I am trying to say.
> > > >>
> > > >> Note that even simple features like user sessions in Falcon will be
> > > coupled
> > > >> with other components in the Hadoop ecosystems. In secure set ups,
> > > Falcon
> > > >> will need to get delegation tokens from the HDFS NameNode and
> perform
> > > >> UGI-related operations. I don't see how Django will help in this
> case.
> > > >>
> > > >> bq. P.S. On a side note, you can use python to interact with Hadoop,
> > > Hive,
> > > >> Pig,
> > > >> Oozie. Again, I am not suggesting to use python instead of java in
> > > existing
> > > >> components.
> > > >>
> > > >> If this is the case, can you elaborate on how Python+Django can
> > benefit
> > > >> Falcon, and what exact requirements need to be solved?
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Sun, Jul 13, 2014 at 9:26 PM, Ajay Yadav <ajaynsit@gmail.com>
> > wrote:
> > > >>
> > > >>> Hi Haohui,
> > > >>>
> > > >>> Thanks for the reply. I am not suggesting to change existing java
> > code
> > > to
> > > >>> python or saying that we can live without javascript. All I am
> > > suggesting
> > > >>> is that we do need to use a server side web framework. Try
> > > implementing a
> > > >>> very basic requirement: user sessions( a logged in user doesn't
> need
> > to
> > > >>> login again on every page) with existing stack and you will
> > understand
> > > >> what
> > > >>> I am trying to say.
> > > >>>
> > > >>>
> > > >>> P.S. On a side note, you can use python to interact with Hadoop,
> > Hive,
> > > >> Pig,
> > > >>> Oozie. Again, I am not suggesting to use python instead of java
in
> > > >> existing
> > > >>> components.
> > > >>>
> > > >>>
> > > >>> On Sun, Jul 13, 2014 at 11:40 PM, Haohui Mai <hmai@hortonworks.com
> >
> > > >> wrote:
> > > >>>
> > > >>>> Thanks for your interests in contributing in Falcon. I am
one of
> the
> > > >>>> early contributors in Falcon UI, here are my two cents:
> > > >>>>
> > > >>>> Falcon has a browser-server architecture which needs to address
> two
> > > >>>> requirements:
> > > >>>>
> > > >>>> 1. Provide a feature-rich UI (e.g. FALCON-290) to the browser.
> > > >>>>
> > > >>>> 2. The server-side logics have to integrate with existing
Hadoop
> > > >>>> ecosystems (e.g., MapReduce, Hive, Pig, and Oozie) which are
> heavily
> > > >>>> based on Java.
> > > >>>>
> > > >>>> While personally I'm pretty open to any other technologies,
but I
> am
> > > >>>> yet to be convinced that how Python+Django will add any values
to
> > > >>>> address the above requirements. More concretely:
> > > >>>>
> > > >>>> i. JavaScript is (and will continue to be) an essential part
of
> the
> > UI
> > > >>>> to meet requirement (1). The python stack cannot replace the
> > > >>>> JavaScript part on the client side.
> > > >>>>
> > > >>>> ii. Falcon has to rely on Java to interact with other projects
in
> > the
> > > >>>> Hadoop ecosystems, as there are few bindings in other languages
> for
> > > >>>> these projects.
> > > >>>>
> > > >>>>
> > > >>>> Using Python+Django might help attract attentions from the
python
> > > >>>> community, but I think that from the Falcon points of view,
it is
> > much
> > > >>>> more important to keep align with the whole Hadoop ecosystems
> (which
> > > >> are
> > > >>>> the main audiences of the Falcon project). Moreover, I don't
think
> > > that
> > > >>> the
> > > >>>> merits that you mentioned for server-side technologies have
to be
> > > >> coupled
> > > >>>> with Python+Django -- in fact, there are many more implementation
> of
> > > >>>> enterprise
> > > >>>> features like exporting to Excel in Java.
> > > >>>>
> > > >>>> In short, I think that the current choice of Java+REST+JavaScript
> is
> > > >>> quite
> > > >>>> reasonable. Though I'm pretty open to any other technologies,
I'm
> > yet
> > > >> to
> > > >>> be
> > > >>>> convinced that Python+Django can be a superior choice.
> > > >>>>
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Haohui
> > > >>>>
> > > >>>>
> > > >>>> On Sat, Jul 12, 2014 at 7:06 PM, Ajay Yadav <ajaynsit@gmail.com>
> > > >> wrote:
> > > >>>>
> > > >>>>> Any comments? I am awaiting your decision to start working
on
> this.
> > > >>>>>
> > > >>>>>
> > > >>>>> On Fri, Jul 11, 2014 at 11:10 AM, Ajay Yadav <ajaynsit@gmail.com
> >
> > > >>> wrote:
> > > >>>>>
> > > >>>>>> Hi Venkatesh,
> > > >>>>>>
> > > >>>>>> Valid question and I am glad that you gave an opportunity
to
> > > >> discuss
> > > >>>> it.
> > > >>>>> I
> > > >>>>>> have no particular bias towards using django. I have
worked with
> > > >>> client
> > > >>>>>> side templates(not dust.js but more popular alternatives
like
> > > >>> mustache,
> > > >>>>>> handlebars etc.) and client side MV* frameworks like
> backbone.js,
> > > >>>>> angularjs
> > > >>>>>> etc. It is completely possible to use both together,
most common
> > > >>> case.
> > > >>>>>>
> > > >>>>>> *Note: *I looked at the existing code and saw that
the current
> > work
> > > >>> in
> > > >>>>>> dust.js is very less - around15-25 lines(only dust
templates
> part
> > > >> not
> > > >>>>>> html/css). So may I take the liberty of rephrasing
your question
> > > >>>> slightly
> > > >>>>>> as below
> > > >>>>>>
> > > >>>>>> *Question*: Assuming we can choose only one to keep
stack mix to
> > > >>>> minimum
> > > >>>>>> - shall we use pure client side framework(like dust.js)
or pure
> > > >>> server
> > > >>>>>> side framework(like django)?
> > > >>>>>>
> > > >>>>>> *Answer*:
> > > >>>>>> *TLDR Version:*
> > > >>>>>> Server side components offer more flexibility and
extensibility
> > > >> than
> > > >>> a
> > > >>>>>> pure client side framework. Using only client side
stack will
> rule
> > > >>> out
> > > >>>>> use
> > > >>>>>> of database for dashboards, talking to other services,plugins
> etc.
> > > >> by
> > > >>>>> using
> > > >>>>>> just dust.js. Things like hardcoding of rest server
end points,
> > > >>>> graceful
> > > >>>>>> degradation, number of parallel connections available
in browser
> > > >> will
> > > >>>>> limit
> > > >>>>>> pure client side stacks. In other cases it will put
a stress on
> > > >>> falcon
> > > >>>>> REST
> > > >>>>>> API to be modified just for the dashboard. Using python
stack
> > > >> instead
> > > >>>> of
> > > >>>>>> javascript, will help in getting more support from
community.
> > > >>>>>>
> > > >>>>>> *Long Version*
> > > >>>>>>
> > > >>>>>> - *Convenience vs. Possibility* - Server side components
give us
> > > >>>>>> flexibility and extensibility to build for any use
case in
> > > >> future
> > > >>> as
> > > >>>>> well.
> > > >>>>>> Any future feature is possible to be built without
dust.js and
> > > >>>>> bootstrap
> > > >>>>>> but not other way around. Dust.js and bootstrap just
make it
> > > >>> easier
> > > >>>>> to do
> > > >>>>>> lot of things to enhance user experience.
> > > >>>>>> - *Features* - Consider the login page of the dashboard.
We will
> > > >>>> need
> > > >>>>>> a database to store users/ integrate with ldap. We
will need to
> > > >>>>> generate
> > > >>>>>> new random passwords when users forget passwords.
Same for
> > > >>> features
> > > >>>>> like
> > > >>>>>> bookmarks, custom dashboards etc. A lot of features
like this
> > > >>> can't
> > > >>>>> be done
> > > >>>>>> if we are not using any server side component.
> > > >>>>>> - *Sanity of REST API* - REST services of falcon are
not only
> > > >> for
> > > >>>> web
> > > >>>>>> browser consumption. They can be used by command line
clients
> > > >> etc.
> > > >>>> as
> > > >>>>> well.
> > > >>>>>> As the use cases for dashboard will evolve, it will
become more
> > > >>>>> involved to
> > > >>>>>> do things using js alone on client side and it will
put lot of
> > > >>>> stress
> > > >>>>> on
> > > >>>>>> modifying REST API to suit just the client. For example
consider
> > > >>> the
> > > >>>>> use
> > > >>>>>> case of allowing an administrator to edit a process
from an HTML
> > > >>>>> form. REST
> > > >>>>>> API will return plain text/xml and it will be tedious
and slow
> > > >> to
> > > >>>>> parse
> > > >>>>>> everything in javascript side and then recreate it.
Sending a
> > > >> json
> > > >>>> and
> > > >>>>>> accepting json just to cater to web client will be
problem for
> > > >>>>>> - Another use case can be to be able to export the
reports like
> > > >>> SLAs
> > > >>>>>> missed in last quarter, metrics in pdf/excel format
and it will
> > > >>> not
> > > >>>> be
> > > >>>>>> possible to do it on client side and in lack of a
server side
> > > >>>>> component it
> > > >>>>>> will again put stress on REST API to be modified just
for
> > > >>> dashboard
> > > >>>>> UI.
> > > >>>>>> - *Best Practices* - With just dust.js and simple
htmls being
> > > >>> served
> > > >>>>>> through a webserver, we will have to deal with naming
urls with
> > > >>>> .html
> > > >>>>>> extensions, hardcoding the REST end points in each
HTML etc.
> > > >>>>>> - *Browser constraints* - We will have to deal with
browser
> > > >>>>>> constraints like number of parallel connections, cross
site
> > > >>>> requests,
> > > >>>>>> javascript not being enabled etc. This will also restrict
the
> > > >>>>> development.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> - *Community Support* - This is purely my impression
and I might
> > > >>> be
> > > >>>>>> completely wrong here. I see that more python developers
are
> > > >>>> involved
> > > >>>>> with
> > > >>>>>> big data/data science/machine learning eco system
than more
> > > >>>> javascript
> > > >>>>>> developers. Hence it will be easier to find more support
for
> > > >>>>> maintaining
> > > >>>>>> and extending these components in community if we
develop them
> > > >> in
> > > >>>>> django
> > > >>>>>> instead of dust.js.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On Fri, Jul 11, 2014 at 2:28 AM, Seetharam Venkatesh
<
> > > >>>>>> venkatesh@innerzeal.com> wrote:
> > > >>>>>>
> > > >>>>>>> I think we are using bootstrap and dust templates.
I'd want to
> > see
> > > >>> if
> > > >>>> we
> > > >>>>>>> need to introduce more technologies into the mix.
Do we need
> > > >> Python
> > > >>>> and
> > > >>>>>>> Django?
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Thu, Jul 10, 2014 at 6:08 PM, Ajay Yadav <
> ajaynsit@gmail.com>
> > > >>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> Thanks Srikanth and Venkatesh.
> > > >>>>>>>>
> > > >>>>>>>> I had looked at the UI and the graph database
related code
> and I
> > > >>>> will
> > > >>>>>>> reuse
> > > >>>>>>>> existing code where ever available. I checked
for
> compatibility
> > > >> of
> > > >>>>> each
> > > >>>>>>> of
> > > >>>>>>>> the software being used and they are all compatible
with APL
> 2.0
> > > >>>>>>> (Python,
> > > >>>>>>>> Django & Bootstrap).
> > > >>>>>>>>
> > > >>>>>>>> If you can assign the task to me on JIRA also,
it will be
> > > >>>>>>>> great(user:ajayyadava) Also, I got the balsamiq
account for
> > > >>>>>>> falcon(thanks
> > > >>>>>>>> Srikant) and I will be asking all my questions
on mocks there.
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Thu, Jul 10, 2014 at 2:45 PM, Seetharam
Venkatesh <
> > > >>>>>>>> venkatesh@innerzeal.com> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Welcome aboard Ajay. Please let us know
how we can help you
> to
> > > >>> get
> > > >>>>> up
> > > >>>>>>> and
> > > >>>>>>>>> running on this. The REST APIs are documented
at:
> > > >>>>>>>>>
> > > >>> http://falcon.incubator.apache.org/docs/restapi/ResourceList.html
> > > >>>>>>>>>
> > > >>>>>>>>> Thanks!
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> On Thu, Jul 10, 2014 at 7:12 AM, Ajay
Yadav <
> > > >> ajaynsit@gmail.com
> > > >>>>
> > > >>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Hi,
> > > >>>>>>>>>>
> > > >>>>>>>>>> I would like to volunteer for
> > > >>>>>>>>>> https://issues.apache.org/jira/browse/FALCON-37
> > > >>>>>>>>>>
> > > >>>>>>>>>> I have more than 7 years of experience
in building
> > > >> websites. I
> > > >>>>> have
> > > >>>>>>>>>> experience in both client side
> > > >>>>>>> technologies(javascript/HTML5/Bootstrap
> > > >>>>>>>>>> etc.) and server side technologies(Python/Django).
> > > >>>>>>>>>>
> > > >>>>>>>>>> I will need some help with iterating
on the mocks as they
> > > >>> don't
> > > >>>>>>> seem to
> > > >>>>>>>>> be
> > > >>>>>>>>>> complete. I spent couple of hours
on building a couple of
> > > >>> pages
> > > >>>>>>> using
> > > >>>>>>>>>> "dummy" data to get an idea of what
all APIs are required.
> > > >>>> Please
> > > >>>>>>> find
> > > >>>>>>>>> them
> > > >>>>>>>>>> attached. They are just a quick and
dirty prototype and UI
> > > >> is
> > > >>>>> pretty
> > > >>>>>>>> raw
> > > >>>>>>>>> as
> > > >>>>>>>>>> of now, to get quick feedback. I used
Django,HTML5 &
> > > >>> Bootstrap.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Please let me know if I can own this
task.
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> --
> > > >>>>>>>>> Regards,
> > > >>>>>>>>> Venkatesh
> > > >>>>>>>>>
> > > >>>>>>>>> “Perfection (in design) is achieved
not when there is nothing
> > > >>> more
> > > >>>>> to
> > > >>>>>>>> add,
> > > >>>>>>>>> but rather when there is nothing more
to take away.”
> > > >>>>>>>>> - Antoine de Saint-Exupéry
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> --
> > > >>>>>>> Regards,
> > > >>>>>>> Venkatesh
> > > >>>>>>>
> > > >>>>>>> “Perfection (in design) is achieved not when
there is nothing
> > more
> > > >>> to
> > > >>>>> add,
> > > >>>>>>> but rather when there is nothing more to take
away.”
> > > >>>>>>> - Antoine de Saint-Exupéry
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>> --
> > > >>>> CONFIDENTIALITY NOTICE
> > > >>>> NOTICE: This message is intended for the use of the individual
or
> > > >> entity
> > > >>> to
> > > >>>> which it is addressed and may contain information that is
> > > confidential,
> > > >>>> privileged and exempt from disclosure under applicable law.
If the
> > > >> reader
> > > >>>> of this message is not the intended recipient, you are hereby
> > notified
> > > >>> that
> > > >>>> any printing, copying, dissemination, distribution, disclosure
or
> > > >>>> forwarding of this communication is strictly prohibited. If
you
> have
> > > >>>> received this communication in error, please contact the sender
> > > >>> immediately
> > > >>>> and delete it from your system. Thank You.
> > > >>>>
> > > >>>
> > > >>
> > > >> --
> > > >> CONFIDENTIALITY NOTICE
> > > >> NOTICE: This message is intended for the use of the individual or
> > > entity to
> > > >> which it is addressed and may contain information that is
> > confidential,
> > > >> privileged and exempt from disclosure under applicable law. If the
> > > reader
> > > >> of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > >> any printing, copying, dissemination, distribution, disclosure or
> > > >> forwarding of this communication is strictly prohibited. If you have
> > > >> received this communication in error, please contact the sender
> > > immediately
> > > >> and delete it from your system. Thank You.
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > > Venkatesh
> > > >
> > > > “Perfection (in design) is achieved not when there is nothing more to
> > > add,
> > > > but rather when there is nothing more to take away.”
> > > > - Antoine de Saint-Exupéry
> > >
> > >
> >
>
>
>
> --
> Regards,
> Venkatesh
>
> “Perfection (in design) is achieved not when there is nothing more to add,
> but rather when there is nothing more to take away.”
> - Antoine de Saint-Exupéry
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message