hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng, Kai" <kai.zh...@intel.com>
Subject RE: Replacing the JSP web UIs to HTML 5 applications
Date Tue, 29 Oct 2013 12:22:19 GMT
> having /JMX for monitoring integration and a /JSON end point for the UI
IMHO, this makes sense, especially for the long term. JMX interface serves as management console
in admin perspective, WebUI serves as end user interface. Both might share same functionality
codes, but that does not validate we couple them together.

Thanks & regards,
Kai

-----Original Message-----
From: Alejandro Abdelnur [mailto:tucu@cloudera.com] 
Sent: Tuesday, October 29, 2013 8:14 AM
To: hdfs-dev@hadoop.apache.org
Subject: Re: Replacing the JSP web UIs to HTML 5 applications

Isn't using JMX to expose JSON for the web UI misusing JMX?

I would think a more appropriate approach would be having /JMX for monitoring integration
and a /JSON end point for the UI data.

Thanks.


On Mon, Oct 28, 2013 at 4:58 PM, Haohui Mai <hmai@hortonworks.com> wrote:

> Alejandro,
>
> If I understand correctly, that is the exact approach that the new web 
> UI is taking. The new web UI takes the output from JMX and renders 
> them as HTML at the client side.
>
> ~Haohui
>
>
> On Mon, Oct 28, 2013 at 4:18 PM, Alejandro Abdelnur <tucu@cloudera.com
> >wrote:
>
> > Haohui,
> >
> > If you have NN and DNs producing JSON instead HTML, then you can 
> > build JS based web UIs. Take for example Oozie, Oozie produces JSON, 
> > it has a
> built
> > in JS web ui that consumes JSON and Hue has built an external web UI 
> > that also consumes JSON. In the case of Hue UI, Oozie didn't have to 
> > change anything to get that UI and improvements on the Hue UI don't 
> > require changes in Oozie unless it is to produce additional information.
> >
> > hope this clarifies.
> >
> > Thx
> >
> >
> > On Mon, Oct 28, 2013 at 4:06 PM, Haohui Mai <hmai@hortonworks.com>
> wrote:
> >
> > > Echo my comments on HDFS-5402:
> > >
> > > bq. If we're going to remove the old web UI, I think the new web 
> > > UI has to have the same level of unit testing. We shouldn't go 
> > > backwards in terms of unit testing.
> > >
> > > I take a look at TestNamenodeJspHelper / TestDatanodeJspHelper / 
> > > TestClusterJspHelper. It seems to me that we can merge these tests 
> > > with
> > the
> > > unit tests on JMX.
> > >
> > > bq. If we are going to
> > > remove this capability, we need to add some other command-line 
> > > tools to get the same functionality. These tools could use REST if 
> > > we have that, or JMX, but they need to exist before we can 
> > > consider removing the old UI.
> > >
> > > This is a good point. Since all information are available through 
> > > JMX,
> > the
> > > easiest way to approach it is to write some scripts using Node.js. 
> > > The architecture of the new Web UIs is ready for this.
> > >
> > >
> > > On Mon, Oct 28, 2013 at 3:57 PM, Alejandro Abdelnur 
> > > <tucu@cloudera.com
> > > >wrote:
> > >
> > > > Producing JSON would be great. Agree with Colin that we should 
> > > > leave
> > for
> > > > now the current JSP based web ui.
> > > >
> > > > thx
> > > >
> > > >
> > > > On Mon, Oct 28, 2013 at 11:16 AM, Colin McCabe <
> cmccabe@alumni.cmu.edu
> > > > >wrote:
> > > >
> > > > > This is a really interesting project, Haohui.  I think it will 
> > > > > make our web UI much nicer.
> > > > >
> > > > > I have a few concerns about removing the old web UI, however:
> > > > >
> > > > > * If we're going to remove the old web UI, I think the new web 
> > > > > UI
> has
> > > > > to have the same level of unit testing.  We shouldn't go 
> > > > > backwards
> in
> > > > > terms of unit testing.
> > > > >
> > > > > * Most of the deployments of elinks and links out there don't
> support
> > > > > Javascript.  This is just a reality of life when using CentOS 
> > > > > 5 or
> 6,
> > > > > which many users are still using.  I have used "links" to 
> > > > > diagnose problems through the web UI in the past, in systems 
> > > > > where access to the cluster was available only through telnet.  
> > > > > If we are going to remove this capability, we need to add some 
> > > > > other command-line
> tools
> > > > > to get the same functionality.  These tools could use REST if 
> > > > > we
> have
> > > > > that, or JMX, but they need to exist before we can consider
> removing
> > > > > the old UI.
> > > > >
> > > > > best,
> > > > > Colin
> > > > >
> > > > > On Fri, Oct 25, 2013 at 7:31 PM, Haohui Mai 
> > > > > <hmai@hortonworks.com>
> > > > wrote:
> > > > > > Thanks for the reply, Luke. Here I just echo my response 
> > > > > > from the
> > > jira:
> > > > > >
> > > > > > bq. this client-side js only approach, which is less secure

> > > > > > than
> a
> > > > > > progressively enhanced hybrid approach used by YARN. The 
> > > > > > recent
> > gmail
> > > > > > XSS fiasco highlights the issue.
> > > > > >
> > > > > > I'm presenting an informal security analysis to compare the
> > security
> > > of
> > > > > the
> > > > > > old and the new web UIs.
> > > > > >
> > > > > > An attacker launches an XSS attack by injecting malicious 
> > > > > > code
> > which
> > > > are
> > > > > > usually HTML or JavaScript fragments into the web page, so 
> > > > > > that
> the
> > > > > > malicious code can have the same privileges of the web page.
> > > > > >
> > > > > > First, in the scope of XSS attacks, note that the threat 
> > > > > > models
> of
> > > > > > launching XSS attacks on Internet sites Gmail/Linkedin and 
> > > > > > the
> one
> > of
> > > > the
> > > > > > Hadoop UIs are different. They have fundamental different 
> > > > > > sets of
> > > > > external
> > > > > > inputs that the attackers have control to. Internet sites 
> > > > > > have
> > little
> > > > > > control of these inputs. In the case of Gmail / Linkedin, an
> attack
> > > can
> > > > > > send you a crafted e-mail, or put malicious description in 
> > > > > > his / her Linkedin profile. The sets of external inputs are
> *restricted*
> > in
> > > > > > Hadoop UIs. The new web UIs take JMX and WebHDFS as inputs.

> > > > > > The attacker has to launch a XSS attack by:
> > > > > >
> > > > > > * Compromise the jars so that the output of JMX / WebHDFS 
> > > > > > have
> the
> > > > > > malicious code.
> > > > > > * Replace the web UIs completely to include the malicious code.
> > > > > >
> > > > > > In either case *the attacker has to compromise the hadoop 
> > > > > > core or
> > the
> > > > > > namenode*. That means the new web UIs are at least as secure

> > > > > > as
> the
> > > > > hadoop
> > > > > > core, and the namenode machine.
> > > > > >
> > > > > > Second, I argue that using client-side templates are more 
> > > > > > secure
> > than
> > > > the
> > > > > > current JSP-based server-side templates. To defend against 
> > > > > > XSS attacks, both techniques have to filter the external 
> > > > > > inputs at
> > > *every*
> > > > > > possible execution paths. Several facts much be taken into 
> > > > > > plays when evaluating the security of both approaches in
> real-world
> > > > > > environments:
> > > > > >
> > > > > > * The JavaScript libraries used in the new web UIs have 
> > > > > > survived
> in
> > > > > > extremely large-scale production tests. jQuery is used by 
> > > > > > Google
> > and
> > > > > >  Microsoft, bootstrap is used by Twitter, and dust.js is 
> > > > > > used by
> > > > > Linkedin.
> > > > > > All libraries survived from hundreds of thousands of  attack

> > > > > > attempts on a daily basis. I agree that the libraries
> might
> > > > still
> > > > > > be imperfect, but there's no way that we can test the JSP 
> > > > > > web  UIs to achieve the same level of assurances given the 
> > > > > > amount of
> > > > > resources
> > > > > > the community has.
> > > > > > * Client-side templates consolidate all filtering logic in 
> > > > > > one
> > > central
> > > > > > place. Recall that the goal is to filter all external inputs

> > > > > > at
> > every
> > > > > >  execution paths, this is a much more systematic approach
> compared
> > to
> > > > the
> > > > > > server-side templates we have today. It is difficult (if not
> > > > > >  impossible) to do it in a JSP/ASP/PHP application, since 
> > > > > > such
> > > > filtering
> > > > > > can be only achieved via ad-hoc approaches ([1] shows some 

> > > > > > empirical data). Also, HDFS-4901 recently describes a XSS
> > > > vulnerability
> > > > > in
> > > > > > browseDirectory.jsp.
> > > > > >
> > > > > > bq. You'd require proper SSL (not self signed) setup to 
> > > > > > avoid JS injection
> > > > > >
> > > > > > Commodity browsers enforce Same-Origin Policy to defend 
> > > > > > against
> > code
> > > > > > injections. It has nothing to do with what kinds of SSL
> > certificates
> > > > > > you hold.
> > > > > >
> > > > > > bq.  I also have concerns that we commit these changes 
> > > > > > without
> > > matching
> > > > > > unit tests
> > > > > >
> > > > > > The JavaScript code can be automatically tested. The same 
> > > > > > code
> can
> > be
> > > > run
> > > > > > by node.js and the test can compared with pre-defined 
> > > > > > results. It is also possible to write an adapter to use 
> > > > > > Rhino to
> > > > > accomplish
> > > > > > the same task. We can discuss how to integrate them into the

> > > > > > maven test routines in a different thread.
> > > > > >
> > > > > > bq. Client side rendering completely breaks the workflows 
> > > > > > for ops
> > who
> > > > > rely
> > > > > > on text based terminal/emacs/vim browsers (no js support) to

> > > > > > monitor component UI.
> > > > > >
> > > > > > links / elinks (http://elinks.or.cz/) are text-based web
> browsers
> > > that
> > > > > > support JavaScript.
> > > > > >
> > > > > > bq. The priority/requirements for UI in core Hadoop should 
> > > > > > be
> > > security
> > > > > and
> > > > > > correctness, which client side templating cannot address 
> > > > > > properly so far.
> > > > > >
> > > > > > I agree that we should focus on security and correctness. 
> > > > > > The
> > > > paragraphs
> > > > > > above explain that how the architecture of the new UIs makes

> > > > > > the UIs more secure in real-world settings compared to the
> UI
> > > we
> > > > > have
> > > > > > today.
> > > > > >
> > > > > > References:
> > > > > >
> > > > > > 1. A. Yip et al. Improving Application Security with Data 
> > > > > > Flow
> > > > > Assertions.
> > > > > > In SOSP'2009.
> > > > > >
> > > > > >
> > > > > > On Fri, Oct 25, 2013 at 10:02 AM, Luke Lu <llu@apache.org>
> wrote:
> > > > > >
> > > > > >> Echoing my comments on HDFS-3555:
> > > > > >>
> > > > > >> I have concerns with this client-side js only approach,

> > > > > >> which is
> > > less
> > > > > >> secure than a progressively enhanced hybrid approach used

> > > > > >> by
> YARN.
> > > The
> > > > > >> recent gmail XSS fiasco highlights the issue. I also have
> concerns
> > > > that
> > > > > we
> > > > > >> commit these changes without matching unit tests - the fact

> > > > > >> you
> > > cannot
> > > > > >> effectively unit test these changes should tell you 
> > > > > >> something
> > about
> > > > this
> > > > > >> approach.
> > > > > >>
> > > > > >> *Requiring* JS means that an admin cannot turn off js to
> > (partially)
> > > > use
> > > > > >> core Hadoop UI. You'd *require* proper SSL (not self 
> > > > > >> signed)
> setup
> > > to
> > > > > avoid
> > > > > >> JS injection, even if security of js libraries used is 
> > > > > >> perfect,
> > > which
> > > > I
> > > > > >> doubt (search gmail/linkedin XSS). Client side rendering
> > completely
> > > > > breaks
> > > > > >> the workflows for ops who rely on text based 
> > > > > >> terminal/emacs/vim
> > > > browsers
> > > > > >> (no js support) to monitor component UI.
> > > > > >>
> > > > > >> IMO, JS-only rendering belongs to social networking sites

> > > > > >> and/or
> > > SaaS
> > > > > >> front-ends, where full time UI/security specialists 
> > > > > >> babysits UI
> > > > > changes. I
> > > > > >> think eventually most users will use a self servicing UI
in 
> > > > > >> a
> SaaS
> > > > > >> front-end that uses REST/JMX API to get data from back-end
> > > components,
> > > > > >> besides their own app master/service UI. The
> priority/requirements
> > > for
> > > > > UI
> > > > > >> in core Hadoop should be security and correctness, which

> > > > > >> client
> > side
> > > > > >> templating cannot address properly so far.
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Oct 22, 2013 at 3:59 PM, Haohui Mai <
> hmai@hortonworks.com
> > >
> > > > > wrote:
> > > > > >>
> > > > > >> > Hi all,
> > > > > >> >
> > > > > >> >
> > > > > >> > Jing Zhao and I recently have reimplemented the JSP-based

> > > > > >> > web
> > UIs
> > > in
> > > > > >> HTML 5
> > > > > >> > applications (HDFS-5333). Based on our prelimanary

> > > > > >> > testing
> > results
> > > > we
> > > > > >> > believe thst the new web UIs of the namenodes and the

> > > > > >> > datanode
> > are
> > > > > ready
> > > > > >> > for everyday uses.
> > > > > >> >
> > > > > >> > You're more than welcome to try it out on trunk by

> > > > > >> > visiting
> > > http://
> > > > > >> > <namenode>/dfshealth.html
> > > > > >> >
> > > > > >> > There are a number of benefits from this transition.
From 
> > > > > >> > a
> > > > > developer's
> > > > > >> > prospective, the most notable one is *maintainability*:
> > > > > >> >
> > > > > >> > (1) The abstractions between the UI and the core server

> > > > > >> > are
> > > > > well-defined,
> > > > > >> > decoupling the UI and the core hadoop servers.
> > > > > >> >
> > > > > >> > (2) It allows us to deprecate the logic in the JSP
pages. 
> > > > > >> > The
> > old
> > > > web
> > > > > UIs
> > > > > >> > have to duplicate the logic in the JSPs. The logic
is 
> > > > > >> > often
> > > > > out-of-dated
> > > > > >> > and not well-tested, which leads to broken pages and

> > > > > >> > security vulnerabilities(e.g. HDFS-5251, HDFS-5307,

> > > > > >> > HDFS-5308,
> HDFS-5317
> > > and
> > > > > >> > HDFS-4901). The architecture of the new UIs prevent
these 
> > > > > >> > bugs
> > at
> > > > the
> > > > > >> very
> > > > > >> > beginning.
> > > > > >> >
> > > > > >> >
> > > > > >> > I propose that deprecate the old, JSP-based web UIs
in 
> > > > > >> > 2.3. I
> > > opened
> > > > > >> > HDFS-5402 to track the relevant discussions.
> > > > > >> >
> > > > > >> > Your feedbacks are highly appreciated.
> > > > > >> >
> > > > > >> >
> > > > > >> > Sincerely,
> > > > > >> >
> > > > > >> > Haohui
> > > > > >> >
> > > > > >> > --
> > > > > >> > CONFIDENTIALITY NOTICE
> > > > > >> > NOTICE: This message is intended for the use of the

> > > > > >> > individual
> > or
> > > > > entity
> > > > > >> to
> > > > > >> > which it is addressed and may contain information that
is
> > > > > confidential,
> > > > > >> > privileged and exempt from disclosure under applicable

> > > > > >> > law. If
> > the
> > > > > reader
> > > > > >> > of this message is not the intended recipient, you
are 
> > > > > >> > hereby
> > > > notified
> > > > > >> that
> > > > > >> > any printing, copying, dissemination, distribution,

> > > > > >> > disclosure
> > or
> > > > > >> > forwarding of this communication is strictly prohibited.

> > > > > >> > If
> you
> > > have
> > > > > >> > received this communication in error, please contact
the
> sender
> > > > > >> immediately
> > > > > >> > and delete it from your system. Thank You.
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > > --
> > > > > > CONFIDENTIALITY NOTICE
> > > > > > NOTICE: This message is intended for the use of the 
> > > > > > individual or
> > > > entity
> > > > > to
> > > > > > which it is addressed and may contain information that is
> > > confidential,
> > > > > > privileged and exempt from disclosure under applicable law.

> > > > > > If
> the
> > > > reader
> > > > > > of this message is not the intended recipient, you are 
> > > > > > hereby
> > > notified
> > > > > that
> > > > > > any printing, copying, dissemination, distribution, 
> > > > > > disclosure or forwarding of this communication is strictly 
> > > > > > prohibited. If you
> > have
> > > > > > received this communication in error, please contact the 
> > > > > > sender
> > > > > immediately
> > > > > > and delete it from your system. Thank You.
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Alejandro
> > > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is 
> > > confidential, privileged and exempt from disclosure under 
> > > applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby 
> > > notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or 
> > > forwarding of this communication is strictly prohibited. If you 
> > > have received this communication in error, please contact the 
> > > sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
> >
> >
> > --
> > Alejandro
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or 
> entity to which it is addressed and may contain information that is 
> confidential, privileged and exempt from disclosure under applicable 
> law. If the reader of this message is not the intended recipient, you 
> are hereby notified that any printing, copying, dissemination, 
> distribution, disclosure or forwarding of this communication is 
> strictly prohibited. If you have received this communication in error, 
> please contact the sender immediately and delete it from your system. Thank You.
>



--
Alejandro

Mime
View raw message