hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: No edits files in dfs.namenode.edits.dir
Date Mon, 23 May 2016 03:33:17 GMT
Are you absolutely certain you are looking at the right directory? The
NameNode is designed to crash if it cannot persist edits (transactions)
durably.

The "hdfs getconf" utility checks local command classpaths but your service
may be running over a different configuration directory. If you have access
to your NameNode web UI, try visiting its /conf URL and you can see the
actual directories configured for the NameNode's images and edit log.

P.s. The config dfs.namenode.edits.dir usually also defaults to the value
of dfs.namenode.name.dir.

On Mon, 23 May 2016 at 11:13 sky88088 <sky880883368@hotmail.com> wrote:

>
> I use "hdfs -getconf" to check all the relate dirs, and still no edits
> files found.
>
> Thanks Ravi's suggestion. I tried "lsof -grep edit" and "lsof -grep
> hadoop", but still nothing meaningful found.
>
> If really no edits persist to disk, any idea about how to fix this problem?
>
> thanks
> ------------------------------
> Subject: Re: No edits files in dfs.namenode.edits.dir
> To: ravihadoop@gmail.com; sky880883368@hotmail.com
> CC: user@hadoop.apache.org
> From: gurmukh.dhillon@yahoo.com.INVALID
> Date: Sun, 22 May 2016 21:45:50 +1000
>
>
> Have you explicitly configured the parameter "dfs.namenode.edits.dir" ?
> else it default to dfs.namenode.name.dir
>
>
>
> On 20/05/16 4:46 AM, Ravi Prakash wrote:
>
> No! You are probably writing the edits file somewhere still. An `lsof` on
> the namenode process may be more revealing. Obviously this depends on
> configuration, but unless you have some really crazy settings, I'm pretty
> sure the edits would be persisted to disk.
>
>
> On Wed, May 18, 2016 at 2:47 AM, sky88088 <sky880883368@hotmail.com>
> wrote:
>
> Hi,
> I found that there is no edits file in my
> cluster's dfs.namenode.edits.dir.
>
>
> It is a 20-nodes cluster without secondary namenode, and it runs well for
> a long while (more than one year).
>
> However, it's found that no edits file in the dfs.namenode.edits.dir and
> the the fsimage file doesn't update.
>
> Is it meaning that all the metadata of name node only stay in the memory?
>
> Is there a way to fix it? Is there any configuration to control the
> persistence?
>
> Thanks!
>
>
>
> --
> --
> Thanks and Regards
>
> Gurmukh Singh
>
>

Mime
View raw message