hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Haohui Mai <h...@hortonworks.com>
Subject Re: Replacing the JSP web UIs to HTML 5 applications
Date Tue, 29 Oct 2013 18:28:03 GMT
I would like to summarize the discussions so far. It seems that we have
reached two consensus:

1. The new JavaScript-based UI is the right direction to go.
2. For now we should keep the old JSP pages around for compatibility
reasons.

There're some debates on the usages of the JMX / JSON APIs, but this is
orthogonal to switching the UI, thus I consider it as a technical detail.
We can continue the discussions in the public jira.

The new UI has already landed in the trunk, based on the consensus it seems
that we can switch the default UI to the new one shortly. The user can
still access the old web UI using the same URLs.

The only question remain is that who is going to maintain the old web UI.
My answer is that we should leave them as deprecated and focus the effort
on the new web UI.

Thanks,
Haohui



On Tue, Oct 29, 2013 at 5:22 AM, Zheng, Kai <kai.zheng@intel.com> wrote:

> > having /JMX for monitoring integration and a /JSON end point for the UI
> IMHO, this makes sense, especially for the long term. JMX interface serves
> as management console in admin perspective, WebUI serves as end user
> interface. Both might share same functionality codes, but that does not
> validate we couple them together.
>
> Thanks & regards,
> Kai
>
> -----Original Message-----
> From: Alejandro Abdelnur [mailto:tucu@cloudera.com]
> Sent: Tuesday, October 29, 2013 8:14 AM
> To: hdfs-dev@hadoop.apache.org
> Subject: Re: Replacing the JSP web UIs to HTML 5 applications
>
> Isn't using JMX to expose JSON for the web UI misusing JMX?
>
> I would think a more appropriate approach would be having /JMX for
> monitoring integration and a /JSON end point for the UI data.
>
> Thanks.
>
>
> On Mon, Oct 28, 2013 at 4:58 PM, Haohui Mai <hmai@hortonworks.com> wrote:
>
> > Alejandro,
> >
> > If I understand correctly, that is the exact approach that the new web
> > UI is taking. The new web UI takes the output from JMX and renders
> > them as HTML at the client side.
> >
> > ~Haohui
> >
> >
> > On Mon, Oct 28, 2013 at 4:18 PM, Alejandro Abdelnur <tucu@cloudera.com
> > >wrote:
> >
> > > Haohui,
> > >
> > > If you have NN and DNs producing JSON instead HTML, then you can
> > > build JS based web UIs. Take for example Oozie, Oozie produces JSON,
> > > it has a
> > built
> > > in JS web ui that consumes JSON and Hue has built an external web UI
> > > that also consumes JSON. In the case of Hue UI, Oozie didn't have to
> > > change anything to get that UI and improvements on the Hue UI don't
> > > require changes in Oozie unless it is to produce additional
> information.
> > >
> > > hope this clarifies.
> > >
> > > Thx
> > >
> > >
> > > On Mon, Oct 28, 2013 at 4:06 PM, Haohui Mai <hmai@hortonworks.com>
> > wrote:
> > >
> > > > Echo my comments on HDFS-5402:
> > > >
> > > > bq. If we're going to remove the old web UI, I think the new web
> > > > UI has to have the same level of unit testing. We shouldn't go
> > > > backwards in terms of unit testing.
> > > >
> > > > I take a look at TestNamenodeJspHelper / TestDatanodeJspHelper /
> > > > TestClusterJspHelper. It seems to me that we can merge these tests
> > > > with
> > > the
> > > > unit tests on JMX.
> > > >
> > > > bq. If we are going to
> > > > remove this capability, we need to add some other command-line
> > > > tools to get the same functionality. These tools could use REST if
> > > > we have that, or JMX, but they need to exist before we can
> > > > consider removing the old UI.
> > > >
> > > > This is a good point. Since all information are available through
> > > > JMX,
> > > the
> > > > easiest way to approach it is to write some scripts using Node.js.
> > > > The architecture of the new Web UIs is ready for this.
> > > >
> > > >
> > > > On Mon, Oct 28, 2013 at 3:57 PM, Alejandro Abdelnur
> > > > <tucu@cloudera.com
> > > > >wrote:
> > > >
> > > > > Producing JSON would be great. Agree with Colin that we should
> > > > > leave
> > > for
> > > > > now the current JSP based web ui.
> > > > >
> > > > > thx
> > > > >
> > > > >
> > > > > On Mon, Oct 28, 2013 at 11:16 AM, Colin McCabe <
> > cmccabe@alumni.cmu.edu
> > > > > >wrote:
> > > > >
> > > > > > This is a really interesting project, Haohui.  I think it will
> > > > > > make our web UI much nicer.
> > > > > >
> > > > > > I have a few concerns about removing the old web UI, however:
> > > > > >
> > > > > > * If we're going to remove the old web UI, I think the new web
> > > > > > UI
> > has
> > > > > > to have the same level of unit testing.  We shouldn't go
> > > > > > backwards
> > in
> > > > > > terms of unit testing.
> > > > > >
> > > > > > * Most of the deployments of elinks and links out there don't
> > support
> > > > > > Javascript.  This is just a reality of life when using CentOS
> > > > > > 5 or
> > 6,
> > > > > > which many users are still using.  I have used "links" to
> > > > > > diagnose problems through the web UI in the past, in systems
> > > > > > where access to the cluster was available only through telnet.
> > > > > > If we are going to remove this capability, we need to add some
> > > > > > other command-line
> > tools
> > > > > > to get the same functionality.  These tools could use REST if
> > > > > > we
> > have
> > > > > > that, or JMX, but they need to exist before we can consider
> > removing
> > > > > > the old UI.
> > > > > >
> > > > > > best,
> > > > > > Colin
> > > > > >
> > > > > > On Fri, Oct 25, 2013 at 7:31 PM, Haohui Mai
> > > > > > <hmai@hortonworks.com>
> > > > > wrote:
> > > > > > > Thanks for the reply, Luke. Here I just echo my response
> > > > > > > from the
> > > > jira:
> > > > > > >
> > > > > > > bq. this client-side js only approach, which is less secure
> > > > > > > than
> > a
> > > > > > > progressively enhanced hybrid approach used by YARN. The
> > > > > > > recent
> > > gmail
> > > > > > > XSS fiasco highlights the issue.
> > > > > > >
> > > > > > > I'm presenting an informal security analysis to compare
the
> > > security
> > > > of
> > > > > > the
> > > > > > > old and the new web UIs.
> > > > > > >
> > > > > > > An attacker launches an XSS attack by injecting malicious
> > > > > > > code
> > > which
> > > > > are
> > > > > > > usually HTML or JavaScript fragments into the web page,
so
> > > > > > > that
> > the
> > > > > > > malicious code can have the same privileges of the web
page.
> > > > > > >
> > > > > > > First, in the scope of XSS attacks, note that the threat
> > > > > > > models
> > of
> > > > > > > launching XSS attacks on Internet sites Gmail/Linkedin
and
> > > > > > > the
> > one
> > > of
> > > > > the
> > > > > > > Hadoop UIs are different. They have fundamental different
> > > > > > > sets of
> > > > > > external
> > > > > > > inputs that the attackers have control to. Internet sites
> > > > > > > have
> > > little
> > > > > > > control of these inputs. In the case of Gmail / Linkedin,
an
> > attack
> > > > can
> > > > > > > send you a crafted e-mail, or put malicious description
in
> > > > > > > his / her Linkedin profile. The sets of external inputs
are
> > *restricted*
> > > in
> > > > > > > Hadoop UIs. The new web UIs take JMX and WebHDFS as inputs.
> > > > > > > The attacker has to launch a XSS attack by:
> > > > > > >
> > > > > > > * Compromise the jars so that the output of JMX / WebHDFS
> > > > > > > have
> > the
> > > > > > > malicious code.
> > > > > > > * Replace the web UIs completely to include the malicious
code.
> > > > > > >
> > > > > > > In either case *the attacker has to compromise the hadoop
> > > > > > > core or
> > > the
> > > > > > > namenode*. That means the new web UIs are at least as secure
> > > > > > > as
> > the
> > > > > > hadoop
> > > > > > > core, and the namenode machine.
> > > > > > >
> > > > > > > Second, I argue that using client-side templates are more
> > > > > > > secure
> > > than
> > > > > the
> > > > > > > current JSP-based server-side templates. To defend against
> > > > > > > XSS attacks, both techniques have to filter the external
> > > > > > > inputs at
> > > > *every*
> > > > > > > possible execution paths. Several facts much be taken into
> > > > > > > plays when evaluating the security of both approaches in
> > real-world
> > > > > > > environments:
> > > > > > >
> > > > > > > * The JavaScript libraries used in the new web UIs have
> > > > > > > survived
> > in
> > > > > > > extremely large-scale production tests. jQuery is used
by
> > > > > > > Google
> > > and
> > > > > > >  Microsoft, bootstrap is used by Twitter, and dust.js is
> > > > > > > used by
> > > > > > Linkedin.
> > > > > > > All libraries survived from hundreds of thousands of  attack
> > > > > > > attempts on a daily basis. I agree that the libraries
> > might
> > > > > still
> > > > > > > be imperfect, but there's no way that we can test the JSP
> > > > > > > web  UIs to achieve the same level of assurances given
the
> > > > > > > amount of
> > > > > > resources
> > > > > > > the community has.
> > > > > > > * Client-side templates consolidate all filtering logic
in
> > > > > > > one
> > > > central
> > > > > > > place. Recall that the goal is to filter all external inputs
> > > > > > > at
> > > every
> > > > > > >  execution paths, this is a much more systematic approach
> > compared
> > > to
> > > > > the
> > > > > > > server-side templates we have today. It is difficult (if
not
> > > > > > >  impossible) to do it in a JSP/ASP/PHP application, since
> > > > > > > such
> > > > > filtering
> > > > > > > can be only achieved via ad-hoc approaches ([1] shows some
> > > > > > > empirical data). Also, HDFS-4901 recently describes a XSS
> > > > > vulnerability
> > > > > > in
> > > > > > > browseDirectory.jsp.
> > > > > > >
> > > > > > > bq. You'd require proper SSL (not self signed) setup to
> > > > > > > avoid JS injection
> > > > > > >
> > > > > > > Commodity browsers enforce Same-Origin Policy to defend
> > > > > > > against
> > > code
> > > > > > > injections. It has nothing to do with what kinds of SSL
> > > certificates
> > > > > > > you hold.
> > > > > > >
> > > > > > > bq.  I also have concerns that we commit these changes
> > > > > > > without
> > > > matching
> > > > > > > unit tests
> > > > > > >
> > > > > > > The JavaScript code can be automatically tested. The same
> > > > > > > code
> > can
> > > be
> > > > > run
> > > > > > > by node.js and the test can compared with pre-defined
> > > > > > > results. It is also possible to write an adapter to use
> > > > > > > Rhino to
> > > > > > accomplish
> > > > > > > the same task. We can discuss how to integrate them into
the
> > > > > > > maven test routines in a different thread.
> > > > > > >
> > > > > > > bq. Client side rendering completely breaks the workflows
> > > > > > > for ops
> > > who
> > > > > > rely
> > > > > > > on text based terminal/emacs/vim browsers (no js support)
to
> > > > > > > monitor component UI.
> > > > > > >
> > > > > > > links / elinks (http://elinks.or.cz/) are text-based web
> > browsers
> > > > that
> > > > > > > support JavaScript.
> > > > > > >
> > > > > > > bq. The priority/requirements for UI in core Hadoop should
> > > > > > > be
> > > > security
> > > > > > and
> > > > > > > correctness, which client side templating cannot address
> > > > > > > properly so far.
> > > > > > >
> > > > > > > I agree that we should focus on security and correctness.
> > > > > > > The
> > > > > paragraphs
> > > > > > > above explain that how the architecture of the new UIs
makes
> > > > > > > the UIs more secure in real-world settings compared to
the
> > UI
> > > > we
> > > > > > have
> > > > > > > today.
> > > > > > >
> > > > > > > References:
> > > > > > >
> > > > > > > 1. A. Yip et al. Improving Application Security with Data
> > > > > > > Flow
> > > > > > Assertions.
> > > > > > > In SOSP'2009.
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Oct 25, 2013 at 10:02 AM, Luke Lu <llu@apache.org>
> > wrote:
> > > > > > >
> > > > > > >> Echoing my comments on HDFS-3555:
> > > > > > >>
> > > > > > >> I have concerns with this client-side js only approach,
> > > > > > >> which is
> > > > less
> > > > > > >> secure than a progressively enhanced hybrid approach
used
> > > > > > >> by
> > YARN.
> > > > The
> > > > > > >> recent gmail XSS fiasco highlights the issue. I also
have
> > concerns
> > > > > that
> > > > > > we
> > > > > > >> commit these changes without matching unit tests -
the fact
> > > > > > >> you
> > > > cannot
> > > > > > >> effectively unit test these changes should tell you
> > > > > > >> something
> > > about
> > > > > this
> > > > > > >> approach.
> > > > > > >>
> > > > > > >> *Requiring* JS means that an admin cannot turn off
js to
> > > (partially)
> > > > > use
> > > > > > >> core Hadoop UI. You'd *require* proper SSL (not self
> > > > > > >> signed)
> > setup
> > > > to
> > > > > > avoid
> > > > > > >> JS injection, even if security of js libraries used
is
> > > > > > >> perfect,
> > > > which
> > > > > I
> > > > > > >> doubt (search gmail/linkedin XSS). Client side rendering
> > > completely
> > > > > > breaks
> > > > > > >> the workflows for ops who rely on text based
> > > > > > >> terminal/emacs/vim
> > > > > browsers
> > > > > > >> (no js support) to monitor component UI.
> > > > > > >>
> > > > > > >> IMO, JS-only rendering belongs to social networking
sites
> > > > > > >> and/or
> > > > SaaS
> > > > > > >> front-ends, where full time UI/security specialists
> > > > > > >> babysits UI
> > > > > > changes. I
> > > > > > >> think eventually most users will use a self servicing
UI in
> > > > > > >> a
> > SaaS
> > > > > > >> front-end that uses REST/JMX API to get data from back-end
> > > > components,
> > > > > > >> besides their own app master/service UI. The
> > priority/requirements
> > > > for
> > > > > > UI
> > > > > > >> in core Hadoop should be security and correctness,
which
> > > > > > >> client
> > > side
> > > > > > >> templating cannot address properly so far.
> > > > > > >>
> > > > > > >>
> > > > > > >> On Tue, Oct 22, 2013 at 3:59 PM, Haohui Mai <
> > hmai@hortonworks.com
> > > >
> > > > > > wrote:
> > > > > > >>
> > > > > > >> > Hi all,
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > Jing Zhao and I recently have reimplemented the
JSP-based
> > > > > > >> > web
> > > UIs
> > > > in
> > > > > > >> HTML 5
> > > > > > >> > applications (HDFS-5333). Based on our prelimanary
> > > > > > >> > testing
> > > results
> > > > > we
> > > > > > >> > believe thst the new web UIs of the namenodes
and the
> > > > > > >> > datanode
> > > are
> > > > > > ready
> > > > > > >> > for everyday uses.
> > > > > > >> >
> > > > > > >> > You're more than welcome to try it out on trunk
by
> > > > > > >> > visiting
> > > > http://
> > > > > > >> > <namenode>/dfshealth.html
> > > > > > >> >
> > > > > > >> > There are a number of benefits from this transition.
From
> > > > > > >> > a
> > > > > > developer's
> > > > > > >> > prospective, the most notable one is *maintainability*:
> > > > > > >> >
> > > > > > >> > (1) The abstractions between the UI and the core
server
> > > > > > >> > are
> > > > > > well-defined,
> > > > > > >> > decoupling the UI and the core hadoop servers.
> > > > > > >> >
> > > > > > >> > (2) It allows us to deprecate the logic in the
JSP pages.
> > > > > > >> > The
> > > old
> > > > > web
> > > > > > UIs
> > > > > > >> > have to duplicate the logic in the JSPs. The logic
is
> > > > > > >> > often
> > > > > > out-of-dated
> > > > > > >> > and not well-tested, which leads to broken pages
and
> > > > > > >> > security vulnerabilities(e.g. HDFS-5251, HDFS-5307,
> > > > > > >> > HDFS-5308,
> > HDFS-5317
> > > > and
> > > > > > >> > HDFS-4901). The architecture of the new UIs prevent
these
> > > > > > >> > bugs
> > > at
> > > > > the
> > > > > > >> very
> > > > > > >> > beginning.
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > I propose that deprecate the old, JSP-based web
UIs in
> > > > > > >> > 2.3. I
> > > > opened
> > > > > > >> > HDFS-5402 to track the relevant discussions.
> > > > > > >> >
> > > > > > >> > Your feedbacks are highly appreciated.
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > Sincerely,
> > > > > > >> >
> > > > > > >> > Haohui
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> > CONFIDENTIALITY NOTICE
> > > > > > >> > NOTICE: This message is intended for the use of
the
> > > > > > >> > individual
> > > or
> > > > > > entity
> > > > > > >> to
> > > > > > >> > which it is addressed and may contain information
that is
> > > > > > confidential,
> > > > > > >> > privileged and exempt from disclosure under applicable
> > > > > > >> > law. If
> > > the
> > > > > > reader
> > > > > > >> > of this message is not the intended recipient,
you are
> > > > > > >> > hereby
> > > > > notified
> > > > > > >> that
> > > > > > >> > any printing, copying, dissemination, distribution,
> > > > > > >> > disclosure
> > > or
> > > > > > >> > forwarding of this communication is strictly prohibited.
> > > > > > >> > If
> > you
> > > > have
> > > > > > >> > received this communication in error, please contact
the
> > sender
> > > > > > >> immediately
> > > > > > >> > and delete it from your system. Thank You.
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > > --
> > > > > > > CONFIDENTIALITY NOTICE
> > > > > > > NOTICE: This message is intended for the use of the
> > > > > > > individual or
> > > > > entity
> > > > > > to
> > > > > > > which it is addressed and may contain information that
is
> > > > confidential,
> > > > > > > privileged and exempt from disclosure under applicable
law.
> > > > > > > If
> > the
> > > > > reader
> > > > > > > of this message is not the intended recipient, you are
> > > > > > > hereby
> > > > notified
> > > > > > that
> > > > > > > any printing, copying, dissemination, distribution,
> > > > > > > disclosure or forwarding of this communication is strictly
> > > > > > > prohibited. If you
> > > have
> > > > > > > received this communication in error, please contact the
> > > > > > > sender
> > > > > > immediately
> > > > > > > and delete it from your system. Thank You.
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Alejandro
> > > > >
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> > > > confidential, privileged and exempt from disclosure under
> > > > applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> > > > notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you
> > > > have received this communication in error, please contact the
> > > > sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > > >
> > >
> > >
> > >
> > > --
> > > Alejandro
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or
> > entity to which it is addressed and may contain information that is
> > confidential, privileged and exempt from disclosure under applicable
> > law. If the reader of this message is not the intended recipient, you
> > are hereby notified that any printing, copying, dissemination,
> > distribution, disclosure or forwarding of this communication is
> > strictly prohibited. If you have received this communication in error,
> > please contact the sender immediately and delete it from your system.
> Thank You.
> >
>
>
>
> --
> Alejandro
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message