hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Forrest Aldrich <for...@gmail.com>
Subject Help: viewing Hadoop web pages via URL to a back-end network, and specifying an alt config directory
Date Mon, 05 Nov 2012 21:59:32 GMT
      I have two questions...

      First, our Hadoop cluster uses a back-end, non-routed network for 
transfering data, it is not routed and not accessible beyond the 
cluster.  Thus, the resulting URLs the Hadoop web pages display are 
inaccessible.

      While looking around, I found a similar, older thread about this at:

http://www.mail-archive.com/core-user@hadoop.apache.org/msg10664.html

     Our developers need to access the URLs to the various job info 
directly -- but I'm not sure how I can do this.   I don't think setting 
dfs.datanode.dns.interface will have any affect.

     I could also NAT a port range from the front-end if I needed to, 
with iptables... but at this point, this is seems cosmetic. Surely 
someone else has run into this :-)  Here's an example of what I'm seeing 
displayed on our hadoop pages:

      The Hadoop web page displays this text:

hadoop.our-backend.com.:hadoop.our-frontend.com/127.0.0.1:42302

      with the actual URL (click) being:

http://hadoop.our-backend.com.:50060/

Note the added "." in the actual links, which is not coming from our 
*.xml config files or /etc/hosts.

The display shows backend -> frontend -> localhost.  Seems a little 
convoluted; at least, I know our developers are getting irritated by it :-)

     There must be a relatively simple solution to this - as surely 
others have a similar configuration?

      My second question is partly due to some confusion on my part from 
reading the documentation.   We want to manage our Hadoop installation 
for each system by moving out the /conf directory to another location.   
There appear to be a few places you can do this, including the command 
line.   What I can't seem to figure out is where (preferrably at 
startup) can I place this value so that it's /inherited/ across the 
board, so I don't have to worry about multiple edits?




Thanks!


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message