Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of marcosluis2186@googlemail.com
 designates 209.85.160.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CACkSZy0335ncn1je-f3-4kNfz5VzpLHStb+T7Khx8ngmngOCbg@mail.gmail.com>
References: 
 <CACkSZy0335ncn1je-f3-4kNfz5VzpLHStb+T7Khx8ngmngOCbg@mail.gmail.com>
From: Marcos Luis Ortiz Valmaseda <marcosluis2186@googlemail.com>
Date: Mon, 28 Nov 2011 14:52:44 -0430
Message-ID: 
 <CAJs-K1sj3DkJJUf2bNV3nvmyXY7S_o73xbFPYTCQKWVKXFOPYw@mail.gmail.com>
Subject: Re: ProtocolProvider errors On MRv2 Failed to use
 org.apache.hadoop.mapred.YarnClientProtocolProvider
To: mapreduce-user@hadoop.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

2011/11/28 Stephen Boesch <javadba@gmail.com>:
>
> Hi
> =C2=A0 I set up a pseudo cluster =C2=A0according to the instructions =C2=
=A0here
> =C2=A0http://www.cloudera.com/blog/2011/11/building-and-deploying-mr2/.
> Initially the randomwriter example worked. But after a crash on the machi=
ne
> and restarting the services I am getting the errors shown below.
> Jps seems to think the processes are running properly:
>
> had@mithril:/shared/hadoop$ jps
> 7980 JobHistoryServer
> 7668 NameNode
> 7821 ResourceManager
> 7748 DataNode
> 8021 Jps
> 7902 NodeManager
>
> $ hadoop jar hadoop-mapreduce-examples-0.23.0.jar =C2=A0randomwriter
> -Dmapreduce.job.user.name=3D$USER
> -Dmapreduce.clientfactory.class.name=3Dorg.apache.hadoop.mapred.YarnClien=
tFactory
> -Dmapreduce.randomwriter.bytespermap=3D10000 -Ddfs.blocksize=3D64m
> -Ddfs.block.size=3D64m -libjars
> $YARN_HOME/modules/hadoop-mapreduce-client-jobclient-0.23.0.jar output
>
> 2011-11-28 10:23:56,102 WARN =C2=A0conf.Configuration
> (Configuration.java:set(629)) - mapred.used.genericoptionsparser is
> deprecated. Instead, use mapreduce.client.genericoptionsparser.used
> 2011-11-28 10:23:56,158 INFO =C2=A0ipc.YarnRPC (YarnRPC.java:create(47)) =
-
> Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 2011-11-28 10:23:56,162 INFO =C2=A0mapred.ResourceMgrDelegate
> (ResourceMgrDelegate.java:<init>(95)) - Connecting to ResourceManager at
> /0.0.0.0:8040
> 2011-11-28 10:23:56,163 INFO =C2=A0ipc.HadoopYarnRPC
> (HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc pr=
oxy
> for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 2011-11-28 10:23:56,203 INFO =C2=A0mapred.ResourceMgrDelegate
> (ResourceMgrDelegate.java:<init>(99)) - Connected to ResourceManager at
> /0.0.0.0:8040
> 2011-11-28 10:23:56,248 INFO =C2=A0mapreduce.Cluster
> (Cluster.java:initialize(116)) - Failed to use
> org.apache.hadoop.mapred.YarnClientProtocolProvider due to error:
> java.lang.reflect.InvocationTargetException
> 2011-11-28 10:23:56,250 INFO =C2=A0mapreduce.Cluster
> (Cluster.java:initialize(111)) - Cannot pick
> org.apache.hadoop.mapred.LocalClientProtocolProvider as the
> ClientProtocolProvider - returned null protocol
> 2011-11-28 10:23:56,251 INFO =C2=A0mapreduce.Cluster
> (Cluster.java:initialize(111)) - Cannot pick
> org.apache.hadoop.mapred.JobTrackerClientProtocolProvider as the
> ClientProtocolProvider - returned null protocol
> java.io.IOException: Cannot initialize Cluster. Please check your
> configuration for mapreduce.framework.name and the correspond server
> addresses.
>
> My =C2=A0*-site.xml files are precisely as shown on the instructions page=
.
> In any case copying here the one that is most germane - mapred-site.xml
> <?xml version=3D"1.0"?>
> <?xml-stylesheet href=3D"configuration.xsl"?>
> <configuration>
> <property>
> <name> mapreduce.framework.name</name>
> <value>yarn</value>
> </property>
> </configuration>
>

Remember that you have to configure two conf files related to Yarn,
 yarn-site.xml:
 <?xml version=3D=E2=80=9D1.0=E2=80=B3?>
<configuration>
<!=E2=80=93 Site specific YARN configuration properties =E2=80=93>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce.shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>

and mapred-site.xml

<?xml version=3D=E2=80=9D1.0=E2=80=B3?>
<?xml-stylesheet href=3D=E2=80=9Dconfiguration.xsl=E2=80=9D?>
<configuration>
<property>
<name> mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>

Regards

--=20
Marcos Luis Ort=C3=ADz Valmaseda
=C2=A0Linux Infrastructure Engineer
=C2=A0Linux User # 418229
=C2=A0http://marcosluis2186.posterous.com
=C2=A0http://www.linkedin.com/in/marcosluis2186
=C2=A0Twitter: @marcosluis2186