hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Why /tmp directory?
Date Fri, 17 Jul 2009 20:30:05 GMT
Hi Akhil,

That's the default configuration, but it's not meant for actual use in a
cluster. You should be manually setting dfs.data.dir, dfs.name.dir, and
mapred.local.dir on your cluster to point to the disks you want Hadoop to
use. The use of /tmp as a default is because it's a convenient location
pretty well guaranteed to be writable on most platforms, so it's handy for
just getting started with minimal configuration.

-Todd

On Fri, Jul 17, 2009 at 12:57 PM, akhil1988 <akhilanger@gmail.com> wrote:

>
> Hi All,
>
> I want to know why do we generally use tmp directory(and not any other) for
> storing hdfs data, knowing the fact that tmp directory is used for storing
> only temporary data?
>
> I was wondering this because when I run a Hbase job on large data, I get
> this DiskErrorExcpetion:
>
>  org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any
> valid local directory for
>
>
> taskTracker/jobcache/job_200904072051_0001/attempt_200904072051_0001_m_000000_1/output/file.out
>
> Please, excuse if this is a naive question.
>
> --Akhil
>
> --
> View this message in context:
> http://www.nabble.com/Why--tmp-directory--tp24540694p24540694.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message