hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Angeles <patr...@cloudera.com>
Subject Re: IOException: Owner 'mapred' for path XY not match expected owner 'AB'
Date Tue, 26 Oct 2010 13:04:23 GMT
Hi Matthias,

Best I can guess, you have uneven permissions on some of your
mapred.local.dir, causing tasks that run using those directories to fail.
See if these are all owned by user:group 'mapred:hadoop', and have
drwxr-xr-x permissions.

Regards,

- Patrick
On Tue, Oct 26, 2010 at 3:34 AM, Mathias Walter <mathias.walter@gmx.net>wrote:

> Hi Guys,
>
> recently I upgraded to the recent Claudera Hadoop distribution. It contains
> hadoop-core-0.20.2+737.jar. If I now run my map job, I
> get the following exception for a few tasks:
>
> java.io.IOException: Owner 'mapred' for path
>
> /hadoop/hdfs5/tmp/taskTracker/mathias.walter/jobcache/job_201010210928_0005/attempt_201010210928_0005_m_000000_0/output/spill437.out
> .index did not match expected owner 'mathias.walter'
>        at
> org.apache.hadoop.io.SecureIOUtils.checkStat(SecureIOUtils.java:182)
>        at
> org.apache.hadoop.io.SecureIOUtils.openForRead(SecureIOUtils.java:108)
>        at org.apache.hadoop.mapred.SpillRecord.<init>(SpillRecord.java:62)
>        at org.apache.hadoop.mapred.SpillRecord.<init>(SpillRecord.java:55)
>        at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1480)
>        at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1172)
>        at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:574)
>        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:641)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:315)
>        at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at javax.security.auth.Subject.doAs(Subject.java:396)
>        at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>        at org.apache.hadoop.mapred.Child.main(Child.java:211)
>
> A total of 8 tasks are running in parallel. They are finished after about 8
> hours, but some of them (19) were crashed with the above
> exception.
>
> Why are so many tasks crashed, but some not?
>
> --
> Kind regards,
> Mathias
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message