Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of sudhakara.st@gmail.com
 designates 209.85.214.174 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAMqmHUvtcaGQwqbTGpbUTWOG7fcYb2-raR4eL8+MgSBxkwNDXg@mail.gmail.com>
References: 
 <CAMqmHUvtcaGQwqbTGpbUTWOG7fcYb2-raR4eL8+MgSBxkwNDXg@mail.gmail.com>
Date: Sun, 28 Apr 2013 01:05:01 +0530
Message-ID: 
 <CAPE0OngEc+-ywvKEBOM7fgpkUigJVi=bqOCC1o0qQOk_Y_0VXA@mail.gmail.com>
Subject: Re: Exception during reduce phase when running jobs remotely
From: sudhakara st <sudhakara.st@gmail.com>
To: user@hadoop.apache.org, orenbum@gmail.com
Content-Type: multipart/alternative; boundary=089e0122976264c54e04db5cbfd5

--089e0122976264c54e04db5cbfd5
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hello,

Check you hadoop.tmp.dir and mapred.local.dir configuration and permissions=
.


On Sat, Apr 27, 2013 at 12:20 AM, Oren Bumgarner <orenbum@gmail.com> wrote:

> I have a small hadoop cluster running 1.0.4 and I'm trying to have it
> setup so that I can run jobs remotely from a computer on the same network
> that is not a part of the cluster. I've got a main java class that
> implements org.apache.hadoop.util.Tool and I'm able to run this job from
> the NameNode using ToolRunner.run(), setting up the JobConf, and submitti=
ng
> with JobClient.submitJob().
>
> When I try to run the same class remotely from any machine that is not th=
e
> NameNode the job is submitted and it appears that the Map tasks
> successfully complete, but I get the following exception for all of the
> reduce tasks:
>
> org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find out=
put/map_0.out in any of the configured local directories
> 	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPa=
thToRead(LocalDirAllocator.java:429)
> 	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAll=
ocator.java:160)
> 	at org.apache.hadoop.mapred.MapOutputFile.getInputFile(MapOutputFile.jav=
a:161)
> 	at org.apache.hadoop.mapred.ReduceTask.getMapFiles(ReduceTask.java:220)
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:398)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma=
tion.java:1121)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)
>
>
> I'm not sure how to interpret that error message. I think I'm missing som=
e config files that are not present on the remote machine but I can't pin d=
own exactly what I need. Does anyone have any guidance on what the above er=
ror means or how to submit jobs remotely?
>
> Thanks,
>
> --Oren
>
>


--=20

Regards,
.....  Sudhakara.st

--089e0122976264c54e04db5cbfd5
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hello,<br><br></div>Check you hadoop.tmp.dir and mapr=
ed.local.dir configuration and permissions.<br><pre><br></pre></div><div cl=
ass=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Sat, Apr 27, 2013=
 at 12:20 AM, Oren Bumgarner <span dir=3D"ltr">&lt;<a href=3D"mailto:orenbu=
m@gmail.com" target=3D"_blank">orenbum@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">I have a small hadoop cluster running 1.0.4 =
and I&#39;m trying to have it setup so that I can run jobs remotely from a =
computer on the same network that is not a part of the cluster. I&#39;ve go=
t a main java class that implements=A0org.apache.hadoop.util.Tool and I&#39=
;m able to run this job from the NameNode using ToolRunner.run(), setting u=
p the JobConf, and submitting with JobClient.submitJob().<div>


<br></div><div>When I try to run the same class remotely from any machine t=
hat is not the NameNode the job is submitted and it appears that the Map ta=
sks successfully complete, but I get the following exception for all of the=
 reduce tasks:</div>


<div><br></div><div><pre>org.apache.hadoop.util.DiskChecker$DiskErrorExcept=
ion: Could not find output/map_0.out in any of the configured local directo=
ries
	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPath=
ToRead(LocalDirAllocator.java:429)
	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAlloc=
ator.java:160)
	at org.apache.hadoop.mapred.MapOutputFile.getInputFile(MapOutputFile.java:=
161)
	at org.apache.hadoop.mapred.ReduceTask.getMapFiles(ReduceTask.java:220)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:398)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformati=
on.java:1121)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)</pre><pre><span sty=
le=3D"font-family:arial;white-space:normal"><br></span></pre><pre><span sty=
le=3D"font-family:arial;white-space:normal">I&#39;m not sure how to interpr=
et that error message. I think I&#39;m missing some config files that are n=
ot present on the remote machine but I can&#39;t pin down exactly what I ne=
ed. Does anyone have any guidance on what the above error means or how to s=
ubmit jobs remotely?</span></pre>


<pre><span style=3D"font-family:arial;white-space:normal">Thanks,</span></p=
re><pre><span style=3D"font-family:arial;white-space:normal">--Oren</span><=
/pre></div>
</blockquote></div><br><br clear=3D"all"><br>-- <br>=A0 =A0 =A0=A0 <br>Rega=
rds,<br>.....=A0 Sudhakara.st<br>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0=A0 <br>
</div>

--089e0122976264c54e04db5cbfd5--