bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Núñez <>
Subject Hue Using MR1?
Date Fri, 06 Dec 2013 09:02:43 GMT

I’m starting an install from BigTop 0.70: HDFS, YARN and Hue, with the goal of building
up from there to a minimal stack. In theory, this should be as simple as ‘yum install hadoop\*
hue\*’; in practice this turns out to be surprisingly broken. For example, after the yum
install hue is reporting a configuration error:

hadoop.mapred_clusters.default.hadoop_mapred_home Current value: /usr/lib/hadoop-0.20-mapreduce
Path does not exist on the filesystem.

Nowhere is this being set from the files that BigTop installed. Why is Hue looking for MR1

Attempting to fix another misconfiguration reported by hue:

hadoop.hdfs_clusters.default.webhdfs_url Current value: None
Failed to access filesystem root

Is simple, however when restarting the daemons with: "for i in hadoop-hdfs-namenode hadoop-hdfs-datanode
; do service $i restart ; done", both daemons fail to restart (say, what happened to
Extracts from the logs show the reasons:

  *   Datanode because: Problem binding to []
Address already in use
  *   Namenode because: Cannot lock storage /var/lib/hadoop-hdfs/cache/hdfs/dfs/name.
The directory is already locked

So what I’ve done is a fresh installed, modified two config files, as described in the hue
configuration section<>, and
tried to restart the HDFS daemons and they’re failing. This probably isn’t what I’d
call ‘working’.

I know that BigTop is at 0.70, but then again version numbers don’t mean much these days.
Is this kind of error to be expected after a fresh install? When will it be safe to assume
that a simple stack (HDFS, YARN, Hue, Oozie, Zookeeper) installation would be working ‘out
of the box’ without loads of manual configuration?

Perhaps I’m missing the point (I’m definitely missing the documentation), but at this
stage it seems BigTop’s primary advantage is to ensure that the collection of packages is
version compatible. Would that be fair to say? I’m not unappreciative of the value there,
but just want to set expectations with people on what they’re getting with BigTop in its
current state. I recall some discussion of configuration before, and there were somewhat different
opinions. One that I recall was (my words, summarizing my understanding):

BigTop doesn’t do any configuration, that’s all left to the individual packages; BigTop
just places them on the filesystem in (somewhat, there’s still .cfg, .ini, .conf files)
a consistent manner.

Would that be a fair statement?

- Steve

View raw message