hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stas Oskin <stas.os...@gmail.com>
Subject Re: Recommended file-system for DataNode
Date Sun, 11 Oct 2009 08:11:32 GMT
Hi.

By the way, about the noatime - is it safe just to set this for all
partitions used, including / and boot?

Thanks.

2009/10/9 Stas Oskin <stas.oskin@gmail.com>

> Hi.
> AFAIK, this space is reserved for root logs, in case the filesystem is
> full, so the kernel won't crash.
>
> From what I seen, it has to be only enabled on the root partition, the data
> partitions it can be safely set to 0.
>
> I usually leave the default 5% on root, boot and swap (as the space savings
> there insignificant), and set to 0 on data partition, where it really gives
> back the 50-60 GB mentioned below.
>
> Regards.
>
>
> 2009/10/9 Edward Capriolo <edlinuxguru@gmail.com>
>
> On a 1tb disk reducing reserved space from 5 to 2 saves almost 30 gb.
>> Cutting the inodes down saves you some space but not nearly as much.
>> Say 10 gb.
>>
>> The differnce is once you format your disk you can't change the inode
>> numbers. Tunefs can tune reserved blocks while the disk is mounted.
>>
>> I did reserved space with tunefs -m2
>> , Noatime then moved on.
>>
>> On 10/9/09, stephen mulcahy <stephen.mulcahy@deri.org> wrote:
>> > paul wrote:
>> >> Check out the bottom of this page:
>> >>
>> >> http://wiki.apache.org/hadoop/DiskSetup
>> >
>> > Just re-reading that page, two suggestions that may not be appropriate,
>> >
>> > 1. Reducing reserved space to 0. AFAIK, ext3 needs a certain amount of
>> > free space to function properly - the man page for mke2fs suggests that
>> > this reserved space is used for defragmentation, as well as being
>> > emergency space reserved for root. A quick Google doesn't turn up
>> > anything more definitive, but setting it to 0 is a bad idea afaics.
>> >
>> > 2. Reducing the number of inodes. This is a good idea, if you are
>> > really, really sure that nothing will create small files on that
>> > partition. Unless you are absolutely certain of this, I would not change
>> > from the default - I'm not clear on how much of an overall saving you'll
>> > make and the downside to running to running out of inodes is that you
>> > start getting "out of space" errors when you try to write to that disk
>> > (despite df showing you loads of free space), so again, I'm not sure I'd
>> > recommend this one.
>> >
>> > -stephen
>> >
>> > --
>> > Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
>> > NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
>> > http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message