hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stas Oskin <stas.os...@gmail.com>
Subject Re: Recommended file-system for DataNode
Date Mon, 12 Oct 2009 09:11:38 GMT
Hi.

Thanks for the advice.

Regards.

2009/10/12 Jason Venner <jason.hadoop@gmail.com>

> Unless you are serving mail via imap or pop, it is generally considered
> safe.
>
> On Sun, Oct 11, 2009 at 1:11 AM, Stas Oskin <stas.oskin@gmail.com> wrote:
>
> > Hi.
> >
> > By the way, about the noatime - is it safe just to set this for all
> > partitions used, including / and boot?
> >
> > Thanks.
> >
> > 2009/10/9 Stas Oskin <stas.oskin@gmail.com>
> >
> > > Hi.
> > > AFAIK, this space is reserved for root logs, in case the filesystem is
> > > full, so the kernel won't crash.
> > >
> > > From what I seen, it has to be only enabled on the root partition, the
> > data
> > > partitions it can be safely set to 0.
> > >
> > > I usually leave the default 5% on root, boot and swap (as the space
> > savings
> > > there insignificant), and set to 0 on data partition, where it really
> > gives
> > > back the 50-60 GB mentioned below.
> > >
> > > Regards.
> > >
> > >
> > > 2009/10/9 Edward Capriolo <edlinuxguru@gmail.com>
> > >
> > > On a 1tb disk reducing reserved space from 5 to 2 saves almost 30 gb.
> > >> Cutting the inodes down saves you some space but not nearly as much.
> > >> Say 10 gb.
> > >>
> > >> The differnce is once you format your disk you can't change the inode
> > >> numbers. Tunefs can tune reserved blocks while the disk is mounted.
> > >>
> > >> I did reserved space with tunefs -m2
> > >> , Noatime then moved on.
> > >>
> > >> On 10/9/09, stephen mulcahy <stephen.mulcahy@deri.org> wrote:
> > >> > paul wrote:
> > >> >> Check out the bottom of this page:
> > >> >>
> > >> >> http://wiki.apache.org/hadoop/DiskSetup
> > >> >
> > >> > Just re-reading that page, two suggestions that may not be
> > appropriate,
> > >> >
> > >> > 1. Reducing reserved space to 0. AFAIK, ext3 needs a certain amount
> of
> > >> > free space to function properly - the man page for mke2fs suggests
> > that
> > >> > this reserved space is used for defragmentation, as well as being
> > >> > emergency space reserved for root. A quick Google doesn't turn up
> > >> > anything more definitive, but setting it to 0 is a bad idea afaics.
> > >> >
> > >> > 2. Reducing the number of inodes. This is a good idea, if you are
> > >> > really, really sure that nothing will create small files on that
> > >> > partition. Unless you are absolutely certain of this, I would not
> > change
> > >> > from the default - I'm not clear on how much of an overall saving
> > you'll
> > >> > make and the downside to running to running out of inodes is that
> you
> > >> > start getting "out of space" errors when you try to write to that
> disk
> > >> > (despite df showing you loads of free space), so again, I'm not sure
> > I'd
> > >> > recommend this one.
> > >> >
> > >> > -stephen
> > >> >
> > >> > --
> > >> > Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
> > >> > NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
> > >> > http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com
> > >> >
> > >>
> > >
> >
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message