hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj K (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3916) various issues with running yarn proxyserver
Date Fri, 30 Mar 2012 12:42:26 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Devaraj K updated MAPREDUCE-3916:
---------------------------------

               Labels: mrv2  (was: )
     Target Version/s: 2.0.0, 3.0.0
    Affects Version/s: 3.0.0
                       2.0.0
               Status: Patch Available  (was: Open)

{quote}Setting yarn.web-proxy.address to match yarn.resourcemanager.address doesn't disable
the proxyserver (althought not setting yarn.web-proxy.address at all correctly disable it
and produces a message: org.apache.hadoop.yarn.YarnException: yarn.web-proxy.address is not
set so the proxy will not run). This contradicts the documentation provided for yarn.web-proxy.address
in yarn-default.xml{quote}
 If we don't set 'yarn.web-proxy.address', then the proxy server will not start as separate
process and start up fails with the message 'org.apache.hadoop.yarn.YarnException: yarn.web-proxy.address
is not set so the proxy will not run' and proxy will run as part of the RM.

 If we give the value for 'yarn.web-proxy.address' which matches to the 'yarn.resourcemanager.address'
then proxy server start up fails with "Address already in use". And also it is not starting
as part of the RM. It is not as per the documentation. I have modified the description for
this property accordingly.

{quote}
Setting yarn.web-proxy.address and running the service results in the following:
{quote}

I have added the 'hadoop-yarn-common\src\main\resources\webapps\proxy' dir with .keep file
in it to resolve this problem as done for other components.
                
> various issues with running yarn proxyserver
> --------------------------------------------
>
>                 Key: MAPREDUCE-3916
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3916
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2, resourcemanager, webapps
>    Affects Versions: 0.23.1, 2.0.0, 3.0.0
>            Reporter: Roman Shaposhnik
>            Assignee: Devaraj K
>              Labels: mrv2
>         Attachments: MAPREDUCE-3916.patch
>
>
> Seem like yarn proxyserver is not operational when running out of the 0.23.1 RC2 tarball.
> # Setting yarn.web-proxy.address to match yarn.resourcemanager.address doesn't disable
the proxyserver (althought not setting yarn.web-proxy.address at all correctly disable it
and produces a message: org.apache.hadoop.yarn.YarnException: yarn.web-proxy.address is not
set so the proxy will not run). This contradicts the documentation provided for yarn.web-proxy.address
in yarn-default.xml
> # Setting yarn.web-proxy.address and running the service results in the following:
> {noformat}
> $ ./sbin/yarn-daemon.sh start proxyserver 
> starting proxyserver, logging to /tmp/hadoop-0.23.1/logs/yarn-rvs-proxyserver-ahmed-laptop.out
> /usr/java/64/jdk1.6.0_22/bin/java -Dproc_proxyserver -Xmx1000m -Dhadoop.log.dir=/tmp/hadoop-0.23.1/logs
-Dyarn.log.dir=/tmp/hadoop-0.23.1/logs -Dhadoop.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.log.file=yarn-rvs-proxyserver-ahmed-laptop.log -Dyarn.home.dir= -Dyarn.id.str=rvs -Dhadoop.root.logger=INFO,DRFA
-Dyarn.root.logger=INFO,DRFA -Djava.library.path=/tmp/hadoop-0.23.1/lib/native -Dyarn.policy.file=hadoop-policy.xml
-Dhadoop.log.dir=/tmp/hadoop-0.23.1/logs -Dyarn.log.dir=/tmp/hadoop-0.23.1/logs -Dhadoop.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.log.file=yarn-rvs-proxyserver-ahmed-laptop.log -Dyarn.home.dir=/tmp/hadoop-0.23.1 -Dhadoop.root.logger=INFO,DRFA
-Dyarn.root.logger=INFO,DRFA -Djava.library.path=/tmp/hadoop-0.23.1/lib/native -classpath
/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/share/hadoop/common/lib/*:/tmp/hadoop-0.23.1/share/hadoop/common/*:/tmp/hadoop-0.23.1/share/hadoop/hdfs:/tmp/hadoop-0.23.1/share/hadoop/hdfs/lib/*:/tmp/hadoop-0.23.1/share/hadoop/hdfs/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/lib/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/lib/*
org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer
> {noformat}
> with the following message found in the logs:
> {noformat}
> 2012-02-24 09:26:31,099 FATAL org.apache.hadoop.yarn.server.webproxy.WebAppProxy: Could
not start proxy web server
> java.io.FileNotFoundException: webapps/proxy not found in CLASSPATH
>         at org.apache.hadoop.http.HttpServer.getWebAppsPath(HttpServer.java:532)
>         at org.apache.hadoop.http.HttpServer.<init>(HttpServer.java:224)
>         at org.apache.hadoop.http.HttpServer.<init>(HttpServer.java:164)
>         at org.apache.hadoop.yarn.server.webproxy.WebAppProxy.start(WebAppProxy.java:85)
>         at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
>         at org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer.main(WebAppProxyServer.java:76)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message