Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 12F37DA5C for ; Wed, 23 Jan 2013 15:39:44 +0000 (UTC) Received: (qmail 16291 invoked by uid 500); 23 Jan 2013 15:39:39 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 15966 invoked by uid 500); 23 Jan 2013 15:39:39 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Delivered-To: moderator for user@hadoop.apache.org Received: (qmail 97929 invoked by uid 99); 23 Jan 2013 15:32:02 -0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of project.linux.proj@gmail.com designates 209.85.216.51 as permitted sender) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=AVQLJFITO0pHnlX+AwgJItv7y9y38/veM6ajjgcP4qY=; b=GgWKIuEV/BjV7FOfghI7b6R65Q2kF1Ivog3NI8tVCbgFxk9azW7lB0VC4U1jdP9Ke4 wA1mfoDB8Pp53mndhmTF2RhgOZ8R0CqcWt0vEUYl530NdD8t+xwa+OJL25cg1vCZIHro 6TFw9ME7uvGQso/2f+wAe/5DBoalDahrXsttfj2VCHQGb+LgTQllwmrTGbW2W0AbOEO+ tmAreR4RRcJtlw0Yuonl6jY7+G7frbNP9OJ6qJIJ7qst634NlWx2+zORL3Sqi1x9l62E zv7QKExzRCvYFVyfBlQ7iKtiMdW5Osk72ePlr26q496Ah7msdEaoXtgwwZAIJ77k4qUK xx/A== MIME-Version: 1.0 X-Received: by 10.229.105.220 with SMTP id u28mr425635qco.111.1358955096510; Wed, 23 Jan 2013 07:31:36 -0800 (PST) In-Reply-To: References: Date: Wed, 23 Jan 2013 21:01:36 +0530 Message-ID: Subject: Re: NameNode low on available disk space From: Mohit Vadhera To: Harsh J Cc: "" Content-Type: multipart/alternative; boundary=002354332912cb107304d3f663b2 X-Virus-Checked: Checked by ClamAV on apache.org --002354332912cb107304d3f663b2 Content-Type: text/plain; charset=ISO-8859-1 Can somebody answer me on this plz ? On Wed, Jan 23, 2013 at 11:44 AM, Mohit Vadhera < project.linux.proj@gmail.com> wrote: > Thanks Guys, As you said the level is already pretty low i.e 100 MB but in > my case the root fs / has 14 G available. What can be the root cause then ? > > /dev/mapper/vg_operamast1-lv_root > 50G 33G 14G 71% / > > As per logs. > 2013-01-21 01:22:52,217 WARN > org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space > available on volume '/dev/mapper/vg_operamast1-lv_root' is 10653696, which > is below the configured reserved amount 104857600 > > > On Wed, Jan 23, 2013 at 11:13 AM, Harsh J wrote: > >> Hi again, >> >> Yes, you need to add it to hdfs-site.xml and restart the NN. >> >> > Thanks Harsh, Do I need to add parameters in hdfs-site.xml and restart >> service namenode. >> > + public static final String DFS_NAMENODE_DU_RESERVED_KEY = >> "dfs.namenode.resource.du. >> reserved"; >> > + public static final long DFS_NAMENODE_DU_RESERVED_DEFAULT = 1024 >> * 1024 * 100; // 100 MB >> >> >> On Wed, Jan 23, 2013 at 10:12 AM, Harsh J wrote: >> >>> Edit your hdfs-site.xml (or whatever place of config your NN uses) to >>> lower the value of property "dfs.namenode.resource.du.reserved". Create a >>> new property if one does not exist, and set the value of space to a >>> suitable level. The default itself is pretty low - 100 MB in bytes. >>> >>> >>> On Wed, Jan 23, 2013 at 9:13 AM, Mohit Vadhera < >>> project.linux.proj@gmail.com> wrote: >>> >>>> Ok Steve. I am forwarding my issue again to the list that you said. The >>>> version is >>>> >>>> Hi, >>>> >>>> Namenode switches into safemode when it has low disk space on the root >>>> fs / i have to manually run a command to leave it. Below are log messages >>>> for low space on root / fs. Is there any parameter so that i can reduce >>>> reserved amount.Hadoop 2.0.0-cdh4.1.2 >>>> >>>> 2013-01-21 01:22:52,217 WARN >>>> org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space >>>> available on volume '/dev/mapper/vg_lv_root' is 10653696, which is below >>>> the configured reserved amount 104857600 >>>> 2013-01-21 01:22:52,218 WARN >>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: NameNode low on >>>> available disk space. Entering safe mode. >>>> 2013-01-21 01:22:52,218 INFO org.apache.hadoop.hdfs.StateChange: STATE* >>>> Safe mode is ON. >>>> >>>> >>>> >>>> On Wed, Jan 23, 2013 at 2:50 AM, Steve Loughran < >>>> steve.loughran@gmail.com> wrote: >>>> >>>>> ser@hadoop.apache.orglist >>>> >>>> >>>> >>>> >>> >>> >>> -- >>> Harsh J >>> >> >> >> >> -- >> Harsh J >> > > --002354332912cb107304d3f663b2 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Can somebody answer me on this plz ?


On Wed, Jan 23, 2013 at= 11:44 AM, Mohit Vadhera <project.linux.proj@gmail.com><= /span> wrote:
Thanks Guys, As you said th= e level is already pretty low i.e 100 MB but=20 in my case the root fs / has 14 G available.=A0 What can be the root cause then ?

/dev/mapper/vg_operamast1-lv_root
=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 50G=A0=A0 33G=A0=A0 14G=A0 71= % /

As per logs.
=A0=A0=A0 2013-01-21 01:22:52,217 WARN=20 org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space=20 available on volume '/dev/mapper/vg_operamast1-lv_root' is 10653696= ,=20 which is below the configured reserved amount 104857600


On Wed, Jan 23, 2013 at 11:13 AM, Harsh J <harsh@clouder= a.com> wrote:
Hi again,

Yes, you need to add it to hdfs-site.xml and restart the NN.

> Tha= nks Harsh, Do I need to add parameters in hdfs-site.xml and restart service= namenode.
> + =A0publi= c static final String =A0DFS_NAMENODE_DU_RESERVED_KEY =3D "dfs.namenod= e.resource.du.
reserved";
> + =A0public static final long =A0 =A0DFS_NAMENODE_DU_RESERVED_DEFAULT = =3D 1024 * 1024 * 100; // 100 MB

=
On Wed, Jan 23, 2013 at 10:12 AM, Harsh J <harsh@cloudera.com> wrote:
Edit your hdfs-site.xml (or= whatever place of config your NN uses) to lower the value of property &quo= t;dfs.namenode.resource.du.reserved". Create a new property if one doe= s not exist, and set the value of space to a suitable level. The default it= self is pretty low - 100 MB in bytes.


On Wed, Jan 23, 2013 at 9:13 AM, Mohit V= adhera <project.linux.proj@gmail.com> wrote:
= Ok Steve. I am forwarding my issue again to the list that you said. The ver= sion is

Hi,

Namenode switches into safemode when it has low disk space on the=20 root fs / i have to manually run a command to leave it. Below are log=20 messages for low space on root / fs. Is there any parameter so that i=20 can reduce reserved amount.Hadoop 2.0.0-cdh4.1.2

2013-01-21 01:22:52,217 WARN=20 org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space=20 available on volume '/dev/mapper/vg_lv_root' is 10653696, which is = below the configured reserved amount 104857600
2013-01-21 01:22:52,218 WARN=20 org.apache.hadoop.hdfs.server.namenode.FSNamesystem: NameNode low on=20 available disk space. Entering safe mode.
2013-01-21 01:22:52,218 INFO org.apache.hadoop.hdfs.StateChange: STATE* Saf= e mode is ON.




On Wed, Jan 23, 2013 at 2:50 AM, Steve Loughran <steve.loughran@gmail.com> wrote:
ser@hadoop.apache.orglist=





<= /div>--
Har= sh J



--
Harsh J


--002354332912cb107304d3f663b2--