hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Boesch <java...@gmail.com>
Subject Re: ProtocolProvider errors On MRv2 Failed to use org.apache.hadoop.mapred.YarnClientProtocolProvider
Date Mon, 28 Nov 2011 21:19:27 GMT
Yes I did both of those already.

2011/11/28 Marcos Luis Ortiz Valmaseda <marcosluis2186@googlemail.com>

> 2011/11/28 Stephen Boesch <javadba@gmail.com>:
> >
> > Hi
> >   I set up a pseudo cluster  according to the instructions  here
> >  http://www.cloudera.com/blog/2011/11/building-and-deploying-mr2/.
> > Initially the randomwriter example worked. But after a crash on the
> machine
> > and restarting the services I am getting the errors shown below.
> > Jps seems to think the processes are running properly:
> >
> > had@mithril:/shared/hadoop$ jps
> > 7980 JobHistoryServer
> > 7668 NameNode
> > 7821 ResourceManager
> > 7748 DataNode
> > 8021 Jps
> > 7902 NodeManager
> >
> > $ hadoop jar hadoop-mapreduce-examples-0.23.0.jar  randomwriter
> > -Dmapreduce.job.user.name=$USER
> > -Dmapreduce.clientfactory.class.name
> =org.apache.hadoop.mapred.YarnClientFactory
> > -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=64m
> > -Ddfs.block.size=64m -libjars
> > $YARN_HOME/modules/hadoop-mapreduce-client-jobclient-0.23.0.jar output
> >
> > 2011-11-28 10:23:56,102 WARN  conf.Configuration
> > (Configuration.java:set(629)) - mapred.used.genericoptionsparser is
> > deprecated. Instead, use mapreduce.client.genericoptionsparser.used
> > 2011-11-28 10:23:56,158 INFO  ipc.YarnRPC (YarnRPC.java:create(47)) -
> > Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> > 2011-11-28 10:23:56,162 INFO  mapred.ResourceMgrDelegate
> > (ResourceMgrDelegate.java:<init>(95)) - Connecting to ResourceManager at
> > /0.0.0.0:8040
> > 2011-11-28 10:23:56,163 INFO  ipc.HadoopYarnRPC
> > (HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc
> proxy
> > for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> > 2011-11-28 10:23:56,203 INFO  mapred.ResourceMgrDelegate
> > (ResourceMgrDelegate.java:<init>(99)) - Connected to ResourceManager at
> > /0.0.0.0:8040
> > 2011-11-28 10:23:56,248 INFO  mapreduce.Cluster
> > (Cluster.java:initialize(116)) - Failed to use
> > org.apache.hadoop.mapred.YarnClientProtocolProvider due to error:
> > java.lang.reflect.InvocationTargetException
> > 2011-11-28 10:23:56,250 INFO  mapreduce.Cluster
> > (Cluster.java:initialize(111)) - Cannot pick
> > org.apache.hadoop.mapred.LocalClientProtocolProvider as the
> > ClientProtocolProvider - returned null protocol
> > 2011-11-28 10:23:56,251 INFO  mapreduce.Cluster
> > (Cluster.java:initialize(111)) - Cannot pick
> > org.apache.hadoop.mapred.JobTrackerClientProtocolProvider as the
> > ClientProtocolProvider - returned null protocol
> > java.io.IOException: Cannot initialize Cluster. Please check your
> > configuration for mapreduce.framework.name and the correspond server
> > addresses.
> >
> > My  *-site.xml files are precisely as shown on the instructions page.
> > In any case copying here the one that is most germane - mapred-site.xml
> > <?xml version="1.0"?>
> > <?xml-stylesheet href="configuration.xsl"?>
> > <configuration>
> > <property>
> > <name> mapreduce.framework.name</name>
> > <value>yarn</value>
> > </property>
> > </configuration>
> >
>
> Remember that you have to configure two conf files related to Yarn,
>  yarn-site.xml:
>  <?xml version=”1.0″?>
> <configuration>
> <!– Site specific YARN configuration properties –>
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> </configuration>
>
> and mapred-site.xml
>
> <?xml version=”1.0″?>
> <?xml-stylesheet href=”configuration.xsl”?>
> <configuration>
> <property>
> <name> mapreduce.framework.name</name>
> <value>yarn</value>
> </property>
> </configuration>
>
> Regards
>
> --
> Marcos Luis Ortíz Valmaseda
>  Linux Infrastructure Engineer
>  Linux User # 418229
>  http://marcosluis2186.posterous.com
>  http://www.linkedin.com/in/marcosluis2186
>  Twitter: @marcosluis2186
>

Mime
View raw message