hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: namenode doesn't start after reboot
Date Thu, 23 Dec 2010 21:08:06 GMT
On Thu, Dec 23, 2010 at 12:47 PM, Jakob Homan <jghoman@gmail.com> wrote:

> Please move discussions of CDH issues to Cloudera's lists.  Thanks.

Hi Jakob,

These bugs are clearly not CDH-specific. NameNode corruption bugs, and best
practices with regard to the storage of NN metadata, are clearly applicable
to any version of Hadoop that users may run, be it Apache, Yahoo, Facebook,
0.20, 0.21, or trunk. If you have reason to believe my suggestion you quoted
below is somehow not relevant to the larger community I would love to hear

My understanding of the ASF goals is that we should encourage a cohesive
community. Asking users of CDH to move general Hadoop questions off of ASF
mailing lists just because of their choice in distros encourages a fractured
community rather than a cohesive one.

Clearly. if a user has a question specifically about Cloudera packaging they
should be directed to the CDH lists so as not to clutter non-CDH users'
inboxes with irrelevant questions. I think if you browse the archives you'll
find that Cloudera employees have been consistent about doing this since we
started the cdh-user list several months ago. But if an issue is a bug that
is likely to occur in trunk, it makes sense to me to leave it on the list
associated with the core project.

Personally I do my best to answer questions on the ASF lists regardless of
which distro the person is using - though our distros have some divergence
in backported patch sets, it's rare that a bug in one distro doesn't allow
us to fix a bug in trunk. I can readily pull up several recent examples of
this, and I'm surprised that there isn't more concern in the general
community about bugs that may result in NN metadata corruption.


> On Thu, Dec 23, 2010 at 12:02 PM, Todd Lipcon <todd@cloudera.com> wrote:
> > On Thu, Dec 23, 2010 at 2:50 AM, Bjoern Schiessle <bjoern@schiessle.org
> >wrote:
> >
> >>
> >> 1. I have set up a second dfs.name.dir which is stored at another
> >> computer (mounted by sshfs)
> >>
> >
> > I would strongly discourage the use of sshfs for the name dir. For one,
> it's
> > slow, and for two, I've sen it have some really weird semantics where
> it's
> > doing write-back caching.
> >
> > Just take a look at its manpage and you should get scared about using it
> for
> > a critical mount point like this.
> >
> > A soft interruptable NFS mount is a much safer bet.
> >
> > -Todd
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >

Todd Lipcon
Software Engineer, Cloudera

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message