hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: Hadoop is ignoring my hadoop.tmp.dir variable
Date Tue, 25 Jan 2011 06:00:42 GMT
If your environment is properly setup to read your configuration files
instead of the defaults in JAR (which is /tmp/hadoop-{user.name}, as
you're seeing it), you should get the right property values in effect.

Also, instead of modifying "hadoop.tmp.dir", you may also
hand-configure "mapred.local.dir" to specify local directories to be
used by tasks, etc.

On Tue, Jan 25, 2011 at 10:15 AM, Christian Stucchio <stucchio@gmail.com> wrote:
> Greetings,
>
> I'm encountering the following problem with hadoo
> p (0.20.2) - for whatever reason, hadoop seems to be completely
> ignoring my hadoop.tmp.dir variable. I expect I've just made some
> stupid mistake, but I'm a bit stuck. Can anyone offer any suggestions?
>
> From my core-site.xml:
>
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
> <configuration>
>  <property>
>    <name>hadoop.tmp.dir</name>
>    <value>/media/hadoop-tmp/</value>
>  </property>
> ...
>
> However, when I run a job, the spill data is dumped to
> /tmp/hadoop-hadoopuser, and printing out the
> configuration.get("hadoop.tmp.dir") reveals that that is indeed the
> temporary directory.
>
> I believe this is the cause of another error I am running into:
>
> 11/01/24 20:46:17 WARN mapred.LocalJobRunner: job_local_0002
> org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
> taskTracker/jobcache/job_local_0002/attempt_local_0002_m_000000_0/output/file.out
> in any of the configured
> local directories
>        at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389)
>        at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
>        at org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:50)
>        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:193)
>
> Can anyone shed some light on this matter?
>
> Thanks,
>
>
> Chris
>



-- 
Harsh J
www.harshj.com

Mime
View raw message