hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Venner <jason.had...@gmail.com>
Subject Re: Recommended file-system for DataNode
Date Mon, 12 Oct 2009 07:37:55 GMT
Unless you are serving mail via imap or pop, it is generally considered
safe.

On Sun, Oct 11, 2009 at 1:11 AM, Stas Oskin <stas.oskin@gmail.com> wrote:

> Hi.
>
> By the way, about the noatime - is it safe just to set this for all
> partitions used, including / and boot?
>
> Thanks.
>
> 2009/10/9 Stas Oskin <stas.oskin@gmail.com>
>
> > Hi.
> > AFAIK, this space is reserved for root logs, in case the filesystem is
> > full, so the kernel won't crash.
> >
> > From what I seen, it has to be only enabled on the root partition, the
> data
> > partitions it can be safely set to 0.
> >
> > I usually leave the default 5% on root, boot and swap (as the space
> savings
> > there insignificant), and set to 0 on data partition, where it really
> gives
> > back the 50-60 GB mentioned below.
> >
> > Regards.
> >
> >
> > 2009/10/9 Edward Capriolo <edlinuxguru@gmail.com>
> >
> > On a 1tb disk reducing reserved space from 5 to 2 saves almost 30 gb.
> >> Cutting the inodes down saves you some space but not nearly as much.
> >> Say 10 gb.
> >>
> >> The differnce is once you format your disk you can't change the inode
> >> numbers. Tunefs can tune reserved blocks while the disk is mounted.
> >>
> >> I did reserved space with tunefs -m2
> >> , Noatime then moved on.
> >>
> >> On 10/9/09, stephen mulcahy <stephen.mulcahy@deri.org> wrote:
> >> > paul wrote:
> >> >> Check out the bottom of this page:
> >> >>
> >> >> http://wiki.apache.org/hadoop/DiskSetup
> >> >
> >> > Just re-reading that page, two suggestions that may not be
> appropriate,
> >> >
> >> > 1. Reducing reserved space to 0. AFAIK, ext3 needs a certain amount of
> >> > free space to function properly - the man page for mke2fs suggests
> that
> >> > this reserved space is used for defragmentation, as well as being
> >> > emergency space reserved for root. A quick Google doesn't turn up
> >> > anything more definitive, but setting it to 0 is a bad idea afaics.
> >> >
> >> > 2. Reducing the number of inodes. This is a good idea, if you are
> >> > really, really sure that nothing will create small files on that
> >> > partition. Unless you are absolutely certain of this, I would not
> change
> >> > from the default - I'm not clear on how much of an overall saving
> you'll
> >> > make and the downside to running to running out of inodes is that you
> >> > start getting "out of space" errors when you try to write to that disk
> >> > (despite df showing you loads of free space), so again, I'm not sure
> I'd
> >> > recommend this one.
> >> >
> >> > -stephen
> >> >
> >> > --
> >> > Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
> >> > NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
> >> > http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com
> >> >
> >>
> >
>



-- 
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message