hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer <awittena...@linkedin.com>
Subject Re: Question about Data Node configuration
Date Wed, 07 Jul 2010 19:17:07 GMT

On Jul 7, 2010, at 10:38 AM, A Levine wrote:
> For the drive that has the OS, only 100G will be used for the OS.  Is
> it good practice to have a partition on the drive that has the OS used
> for the dfs.data.dir?

I've always partitioned out the root drive so that there is a dedicated file system for Hadoop.
 In other words, the root disk has two (or more) mount points.  Just don't use / directly
in the hadoop configs. That's asking for trouble.

>  Will this slow things down?

 Another spindle = more happiness.  

No.  It will speed things up, unless you do a *lot* of heavy streaming. 

>  Will the size difference available to each directory be a problem?  

It shouldn't be.  Most OS partitions are barely a blip.  The system will just think you are
using more map red space there. :)

> Also, if it is not a good idea to use the OS drive, then how about pointing logs to
> that drive?

That would work too, but for the most part, compute node logs are fairly useless until you
need to do deep debugging.  So it is kind of waste of space.
View raw message