Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 399EFE57D for ; Wed, 23 Jan 2013 06:22:38 +0000 (UTC) Received: (qmail 73729 invoked by uid 500); 23 Jan 2013 06:22:33 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 73452 invoked by uid 500); 23 Jan 2013 06:22:33 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Delivered-To: moderator for user@hadoop.apache.org Received: (qmail 56744 invoked by uid 99); 23 Jan 2013 06:14:37 -0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of project.linux.proj@gmail.com designates 209.85.128.46 as permitted sender) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=S3wuhc69TFJmNeCZy56LbxgdToirreSb4zCB/Xh5iqA=; b=uBKt0j0uxXjIi8MutXvz38xTGC3oqe9CGqZv82CHgegOOkjF9oaI7BNx/jWKQxjdaH /t7haa+CBrB++ShbQsuWq8b9VV/DrkhHI4BXkZ16vJoFgquxI/VxI18R0SlnpBKrz3s0 7qvuKBK8Sdv11DauUlCX1a+auLQC1wKEh5hM69vubDmJdEI8PpvjAF+c90/9sgqSZ48O li0YbISMHkv1K28Bhh4eWlrin/pkU631kluBMFMM4QtY3VLdN59xF4PHn+Hxyc3viTCw mqpHuoUXoEblzIsEonbH3gGvwI/hZOla9bLhrzd3xdNxNbwHBU9rHCYklSUjYJXO0jjZ UIKA== MIME-Version: 1.0 X-Received: by 10.229.178.77 with SMTP id bl13mr33788qcb.130.1358921650406; Tue, 22 Jan 2013 22:14:10 -0800 (PST) In-Reply-To: References: Date: Wed, 23 Jan 2013 11:44:10 +0530 Message-ID: Subject: Re: NameNode low on available disk space From: Mohit Vadhera To: Harsh J Cc: "" Content-Type: multipart/alternative; boundary=0016e68f9f6040061404d3ee9a60 X-Virus-Checked: Checked by ClamAV on apache.org --0016e68f9f6040061404d3ee9a60 Content-Type: text/plain; charset=ISO-8859-1 Thanks Guys, As you said the level is already pretty low i.e 100 MB but in my case the root fs / has 14 G available. What can be the root cause then ? /dev/mapper/vg_operamast1-lv_root 50G 33G 14G 71% / As per logs. 2013-01-21 01:22:52,217 WARN org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space available on volume '/dev/mapper/vg_operamast1-lv_root' is 10653696, which is below the configured reserved amount 104857600 On Wed, Jan 23, 2013 at 11:13 AM, Harsh J wrote: > Hi again, > > Yes, you need to add it to hdfs-site.xml and restart the NN. > > > Thanks Harsh, Do I need to add parameters in hdfs-site.xml and restart > service namenode. > > + public static final String DFS_NAMENODE_DU_RESERVED_KEY = > "dfs.namenode.resource.du. > reserved"; > > + public static final long DFS_NAMENODE_DU_RESERVED_DEFAULT = 1024 * > 1024 * 100; // 100 MB > > > On Wed, Jan 23, 2013 at 10:12 AM, Harsh J wrote: > >> Edit your hdfs-site.xml (or whatever place of config your NN uses) to >> lower the value of property "dfs.namenode.resource.du.reserved". Create a >> new property if one does not exist, and set the value of space to a >> suitable level. The default itself is pretty low - 100 MB in bytes. >> >> >> On Wed, Jan 23, 2013 at 9:13 AM, Mohit Vadhera < >> project.linux.proj@gmail.com> wrote: >> >>> Ok Steve. I am forwarding my issue again to the list that you said. The >>> version is >>> >>> Hi, >>> >>> Namenode switches into safemode when it has low disk space on the root >>> fs / i have to manually run a command to leave it. Below are log messages >>> for low space on root / fs. Is there any parameter so that i can reduce >>> reserved amount.Hadoop 2.0.0-cdh4.1.2 >>> >>> 2013-01-21 01:22:52,217 WARN >>> org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space >>> available on volume '/dev/mapper/vg_lv_root' is 10653696, which is below >>> the configured reserved amount 104857600 >>> 2013-01-21 01:22:52,218 WARN >>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: NameNode low on >>> available disk space. Entering safe mode. >>> 2013-01-21 01:22:52,218 INFO org.apache.hadoop.hdfs.StateChange: STATE* >>> Safe mode is ON. >>> >>> >>> >>> On Wed, Jan 23, 2013 at 2:50 AM, Steve Loughran < >>> steve.loughran@gmail.com> wrote: >>> >>>> ser@hadoop.apache.orglist >>> >>> >>> >>> >> >> >> -- >> Harsh J >> > > > > -- > Harsh J > --0016e68f9f6040061404d3ee9a60 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Thanks Guys, As you said the level is already pretty low i= .e 100 MB but=20 in my case the root fs / has 14 G available.=A0 What can be the root cause then ?

/dev/mapper/vg_operamast1-lv_root
=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 50G=A0=A0 33G=A0=A0 14G=A0 71= % /

As per logs.
=A0=A0=A0 2013-01-21 01:22:52,217 WARN=20 org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space=20 available on volume '/dev/mapper/vg_operamast1-lv_root' is 10653696= ,=20 which is below the configured reserved amount 104857600


On Wed, Jan 23, 2013 at= 11:13 AM, Harsh J <harsh@cloudera.com> wrote:
Hi again,

Yes, you need to add it to hdfs-site.xml and restart the NN.

> Tha= nks Harsh, Do I need to add parameters in hdfs-site.xml and restart service= namenode.
> + =A0publi= c static final String =A0DFS_NAMENODE_DU_RESERVED_KEY =3D "dfs.namenod= e.resource.du.
reserved";
> + =A0public static final long =A0 =A0DFS_NAMENODE_DU_RESERVED_DEFAULT = =3D 1024 * 1024 * 100; // 100 MB

=
On Wed, Jan 23, 2013 at 10:12 AM, Harsh J <harsh@cloudera.com> wrote:
Edit your hdfs-site.xml (or= whatever place of config your NN uses) to lower the value of property &quo= t;dfs.namenode.resource.du.reserved". Create a new property if one doe= s not exist, and set the value of space to a suitable level. The default it= self is pretty low - 100 MB in bytes.


On Wed, Jan 23, 2013 at 9:13 AM, Mohit V= adhera <project.linux.proj@gmail.com> wrote:
= Ok Steve. I am forwarding my issue again to the list that you said. The ver= sion is

Hi,

Namenode switches into safemode when it has low disk space on the=20 root fs / i have to manually run a command to leave it. Below are log=20 messages for low space on root / fs. Is there any parameter so that i=20 can reduce reserved amount.Hadoop 2.0.0-cdh4.1.2

2013-01-21 01:22:52,217 WARN=20 org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space=20 available on volume '/dev/mapper/vg_lv_root' is 10653696, which is = below the configured reserved amount 104857600
2013-01-21 01:22:52,218 WARN=20 org.apache.hadoop.hdfs.server.namenode.FSNamesystem: NameNode low on=20 available disk space. Entering safe mode.
2013-01-21 01:22:52,218 INFO org.apache.hadoop.hdfs.StateChange: STATE* Saf= e mode is ON.




On Wed, Jan 23, 2013 at 2:50 AM, Steve Loughran <steve.loughran@gmail.com> wrote:
ser@hadoop.apache.orglist=





<= /div>--
Harsh J



--
Harsh J

--0016e68f9f6040061404d3ee9a60--