Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9E63110A42 for ; Sun, 15 Dec 2013 10:49:36 +0000 (UTC) Received: (qmail 48621 invoked by uid 500); 15 Dec 2013 10:49:27 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 48512 invoked by uid 500); 15 Dec 2013 10:49:26 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 48505 invoked by uid 99); 15 Dec 2013 10:49:25 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 15 Dec 2013 10:49:25 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of hegman12@gmail.com designates 209.85.214.195 as permitted sender) Received: from [209.85.214.195] (HELO mail-ob0-f195.google.com) (209.85.214.195) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 15 Dec 2013 10:49:20 +0000 Received: by mail-ob0-f195.google.com with SMTP id gq1so1059538obb.6 for ; Sun, 15 Dec 2013 02:48:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=36YZT7dY2TWsOmT9papGBbUuYORdXn81bvw+4v3y2pI=; b=b9osjJKFXFIvo6zMpHLiKk/McGyzs3hU1btJ5GZXh6zmmEPAVRHg2jGXwuetD+qljY L3Eqx6dkM3rWxMd4Y7ROKuREZiSUPZ8j2qP2nhzdEYK5nJV9F4DONyM3D2UbOPmw1EoE rLjUrp4mD+q0Y5H2gkPdFSqtJnaY+MXTnUD62z72ycMnD2w3iDR9UFizXxsaPhCRtzfJ FEb2LvTdt0Cd14yVISoQAam7avWP8F/kxWYR4QfKVG4Yy2XefJjtlDkJsV1Q6OsiFFkF wpw/l44yJc4pQ6YFmGYXuVvKrEz3NB2P6Va2b6y16WDSqfu7x+KXW96i4rBkrVJHU/eF 4AmA== MIME-Version: 1.0 X-Received: by 10.60.74.37 with SMTP id q5mr7916992oev.3.1387104539586; Sun, 15 Dec 2013 02:48:59 -0800 (PST) Received: by 10.182.113.229 with HTTP; Sun, 15 Dec 2013 02:48:59 -0800 (PST) Date: Sun, 15 Dec 2013 16:18:59 +0530 Message-ID: Subject: OOM error and then system hangs From: Manjunath Hegde To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a1135f7bc5934d604ed907166 X-Virus-Checked: Checked by ClamAV on apache.org --001a1135f7bc5934d604ed907166 Content-Type: text/plain; charset=ISO-8859-1 Hi, I am just trying to run wordcount job on 128KB file. First it failed with java space error. Then I have allocated around 2GB space for child JVMs. Now the system just hangs and have to restart the machine. 1. I am running ubuntu 13, 4GB RAM. 2. Datanodes reside on 2 virtual machines in virtualbox, both 1GB RAM and ubuntu server(latest) Another problem is i am getting beloe exception in nodemanager log. 2013-12-15 15:52:03,216 FATAL org.apache.hadoop.yarn.server.nodemanager.NodeManager: Error starting NodeManager org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from master-01 doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:181) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:199) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:339) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:386) Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from master-01 doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:246) at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:175) ... 6 more 2013-12-15 15:52:03,219 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: SHUTDOWN_MSG: An idea?? Thanks, Manjunath --001a1135f7bc5934d604ed907166 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi,
=A0=A0 I a= m just trying to run wordcount job on 128KB file. First it failed with java= space error. Then I have allocated around 2GB space for child JVMs. Now th= e system just hangs and have to restart the machine.

1. I am running ubuntu 13, 4GB RAM.
2. Datanodes reside = on 2 virtual machines in virtualbox, both 1GB RAM and ubuntu server(latest)=


Another problem is i am getting beloe exception in nodema= nager log.

2013-12-15 15:52:03,216 FATAL org.apache.hadoop.yarn.server.nodemanager= .NodeManager: Error starting NodeManager
org.apache.hadoop.yarn.exceptio= ns.YarnRuntimeException: org.apache.hadoop.yarn.exceptions.YarnRuntimeExcep= tion: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeMa= nager failed, Message from ResourceManager: NodeManager from=A0 master-01 d= oesn't satisfy minimum allocations, Sending SHUTDOWN signal to the Node= Manager.
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.yarn.server.nodemanager.NodeStat= usUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:181)
=A0=A0=A0=A0= =A0=A0=A0 at org.apache.hadoop.service.AbstractService.start(AbstractServic= e.java:193)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.service.Composite= Service.serviceStart(CompositeService.java:121)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.yarn.server.nodemanager.NodeMana= ger.serviceStart(NodeManager.java:199)
=A0=A0=A0=A0=A0=A0=A0 at org.apac= he.hadoop.service.AbstractService.start(AbstractService.java:193)
=A0=A0= =A0=A0=A0=A0=A0 at org.apache.hadoop.yarn.server.nodemanager.NodeManager.in= itAndStartNodeManager(NodeManager.java:339)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.yarn.server.nodemanager.NodeMana= ger.main(NodeManager.java:386)
Caused by: org.apache.hadoop.yarn.excepti= ons.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Re= gistration of NodeManager failed, Message from ResourceManager: NodeManager= from=A0 master-01 doesn't satisfy minimum allocations, Sending SHUTDOW= N signal to the NodeManager.
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.yarn.server.nodemanager.NodeStat= usUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:246)
=A0=A0=A0= =A0=A0=A0=A0 at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdater= Impl.serviceStart(NodeStatusUpdaterImpl.java:175)
=A0=A0=A0=A0=A0=A0=A0 ... 6 more
2013-12-15 15:52:03,219 INFO org.apache= .hadoop.yarn.server.nodemanager.NodeManager: SHUTDOWN_MSG:


An idea??

Thanks,
Manjunath
--001a1135f7bc5934d604ed907166--