hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Wilcox <Tom.Wil...@cse.org.uk>
Subject RE: Using Java Remote Method Invocation to make a UI for Hadoop
Date Mon, 09 Jan 2012 16:57:38 GMT
Thanks Harsh J,

I've had a crack at setting up HUE and I now remember why we didn't go for it.

It appears that HUE wants to work with Hadoop 0.20 and we are using Hadoop 1.0 and HBase 0.92...

I don't suppose there are any other efforts like HUE out there that might be more compatible
with our setup? 

Or can anyone see anything wrong with using Grails and RMI to talk to our namenode?

From: Harsh J [harsh@cloudera.com]
Sent: 09 January 2012 14:39
To: Tom Wilcox
Subject: Re: Using Java Remote Method Invocation to make a UI for Hadoop

Hey Tom,

Just wondering, would http://github.com/cloudera/hue have not helped you at all?

On 09-Jan-2012, at 6:59 PM, Tom Wilcox wrote:

> Hi,
> We are trying to make a UI for our HBase + Hadoop applications.
> Basically, what we want is a web front end that can be hosted on a different server and
present data from HBase, as well as allow us to launch Map-Reduce jobs from the web browser.
> Our current approach is as follows:
> We have a 6-node development cluster with each node running Scientific Linux in a VM
with the following configuration:
> Node - Hostname - Daemons
> --------------------------------------------
> Node 1 - "namenode" - namenode, secondarynamenode, regionserver, hbase-master, zookeeper
> Node 2 - "jobtracker" - jobtracker, zookeeper
> Node 3 - "slave0" - zookeeper, datanode, tasktracker
> Node 4 - "slave1" - datanode, tasktracker
> Node 5 - "slave2" - datanode, tasktracker
> Node 6 - "slave3" - datanode, tasktracker
> We have a 7th scientific linux VM running a Grails web application called "BillyWeb".
> We have created several Map-Reduce applications that run on the namenode to process HBase
data and populate a results table. Currently, these MR apps are run from the command line
on namenode.
> We use the HBase REST interface to query data from the results table and present it in
an AJAX-enabled web page. That is the reading part of the problem solved :)
> Now we need to have a HTML page with a button, that when clicked will execute an MR app
on the namenode. This forms the writing part of the problem.
> Currently, we are attempting to do this using Java's Remote Method Invocation (RMI) feature.
I have successfully created a Java client-server application where the server program runs
on "namenode" and the client runs on "BillyWeb" successfully. The client program calls a remote
object method to trigger the MR job on namenode and gets the result.
> We are now in the process of integrating it into the Grails webapp, by adding the source
code and calling it using groovy taglibs. The only remaining issue we are wrestling with is
the need to modify the security policy of the web app to allow access to the client portions
of the source code to the remote server.
> Eventually we hope to have an internal web site that you can visit in any web browser
to view custom-made visualisations of the data and execute parameterised MR jobs to process
data stored in HBase (amongst other places).
> We would like to know your thoughts on this approach. In particular, this feels like
a potentially convoluted approach to building a front-end that may feature several redundant
steps (are we are reinventing too many wheels?).
> Is there an alternative approach that you can think of that might be more sensible?
> Can you think of any problems with the approach we are currently taking?
> Thanks,
> Tom

View raw message