Return-Path: Delivered-To: apmail-hadoop-chukwa-user-archive@minotaur.apache.org Received: (qmail 58012 invoked from network); 27 May 2010 17:40:51 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 27 May 2010 17:40:51 -0000 Received: (qmail 49440 invoked by uid 500); 27 May 2010 17:40:51 -0000 Delivered-To: apmail-hadoop-chukwa-user-archive@hadoop.apache.org Received: (qmail 49416 invoked by uid 500); 27 May 2010 17:40:51 -0000 Mailing-List: contact chukwa-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: chukwa-user@hadoop.apache.org Delivered-To: mailing list chukwa-user@hadoop.apache.org Received: (qmail 49406 invoked by uid 99); 27 May 2010 17:40:51 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 May 2010 17:40:51 +0000 X-ASF-Spam-Status: No, hits=-3.0 required=10.0 tests=AWL,RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of Alan.Ratner@ngc.com designates 155.104.240.104 as permitted sender) Received: from [155.104.240.104] (HELO xmrm0101.northgrum.com) (155.104.240.104) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 May 2010 17:40:44 +0000 Received: from xbhm0001.northgrum.com ([155.104.118.90]) by xmrm0101.northgrum.com with InterScan Message Security Suite; Thu, 27 May 2010 13:35:44 -0400 Received: from XBHIL103.northgrum.com ([134.223.165.23]) by xbhm0001.northgrum.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Thu, 27 May 2010 13:40:23 -0400 Received: from XMBIL132.northgrum.com ([134.223.166.142]) by XBHIL103.northgrum.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Thu, 27 May 2010 12:40:06 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: Chukwa Installation Problems Date: Thu, 27 May 2010 12:40:05 -0500 Message-ID: <68F06E0D1DB9E64A9DB52608086F76B860146A@XMBIL132.northgrum.com> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Chukwa Installation Problems Thread-Index: Acr8RRT2DTocwX6RRuW5eHQY8u6v8ABT1yFQAAhGu/oAAayJEAAAhkiZAAFG8VA= References: <68F06E0D1DB9E64A9DB52608086F76B8601462@XMBIL132.northgrum.com> From: "Ratner, Alan S (IS)" To: X-OriginalArrivalTime: 27 May 2010 17:40:06.0698 (UTC) FILETIME=[A54960A0:01CAFDC3] Thank you Jerome. This part seems to work now. ngc@hadoop1:~/hadoop-0.20.2$ bin/hadoop namenode -format 10/05/27 13:10:49 INFO namenode.NameNode: STARTUP_MSG:=20 /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host =3D hadoop1/10.64.147.2 STARTUP_MSG: args =3D [-format] STARTUP_MSG: version =3D 0.20.2 STARTUP_MSG: build =3D https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010 ************************************************************/ Re-format filesystem in /tmp/hadoop-ngc/dfs/name ? (Y or N) Y 10/05/27 13:10:54 INFO namenode.FSNamesystem: fsOwner=3Dngc,ngc,adm,dialout,cdrom,plugdev,admin,lpadmin,sambashare 10/05/27 13:10:54 INFO namenode.FSNamesystem: supergroup=3Dsupergroup 10/05/27 13:10:54 INFO namenode.FSNamesystem: isPermissionEnabled=3Dtrue 10/05/27 13:10:54 INFO log4j.Log4JMetricsContext: Log4JMetricsContext.dfs.period=3D60 10/05/27 13:10:54 INFO log4j.Log4JMetricsContext: Log4JMetricsContext.dfs.directory=3D/tmp/chukwa/log/metrics/ 10/05/27 13:10:54 INFO log4j.Log4JMetricsContext: Log4JMetricsContext.dfs.uuid has been activated. 10/05/27 13:10:54 INFO common.Storage: Image file of size 93 saved in 0 seconds. 10/05/27 13:10:54 INFO common.Storage: Storage directory /tmp/hadoop-ngc/dfs/name has been successfully formatted. 10/05/27 13:10:54 INFO namenode.NameNode: SHUTDOWN_MSG:=20 /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at hadoop1/10.64.147.2 ************************************************************/ -----Original Message----- From: Jerome Boulon [mailto:jboulon@netflix.com]=20 Sent: Thursday, May 27, 2010 1:02 PM To: chukwa-user@hadoop.apache.org Subject: Re: Chukwa Installation Problems My bad, You also need json.jar in hadoop/lib/ directory. /Jerome On 5/27/10 9:58 AM, "Ratner, Alan S (IS)" wrote: > Thanks for the tip. This detail seems to have been omitted from > http://www.cs.berkeley.edu/~asrabkin/chukwa/admin.html > and http://wiki.apache.org/hadoop/Chukwa_Console_Integration_Guide. So > I copied chukwa-0.4.0/chukwa-hadoop-0.4.0-client.jar (the closest name > match I could get for chukwa-client-.jar) to all the hadoop/lib > directories in my cluster and copied log4j.properties and > hadoop-metrics.properties to all the hadoop/conf directories in my > cluster. But now when I reformat the namenode I get a different error: >=20 > ngc@hadoop1:~/hadoop-0.20.2$ bin/hadoop namenode -format > 10/05/27 12:45:17 INFO namenode.NameNode: STARTUP_MSG: > /************************************************************ > STARTUP_MSG: Starting NameNode > STARTUP_MSG: host =3D hadoop1/10.64.147.2 > STARTUP_MSG: args =3D [-format] > STARTUP_MSG: version =3D 0.20.2 > STARTUP_MSG: build =3D > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r > 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010 > ************************************************************/ > Re-format filesystem in /tmp/hadoop-ngc/dfs/name ? (Y or N) Y > 10/05/27 12:45:20 INFO namenode.FSNamesystem: > fsOwner=3Dngc,ngc,adm,dialout,cdrom,plugdev,admin,lpadmin,sambashare > 10/05/27 12:45:20 INFO namenode.FSNamesystem: supergroup=3Dsupergroup > 10/05/27 12:45:20 INFO namenode.FSNamesystem: = isPermissionEnabled=3Dtrue > 10/05/27 12:45:20 ERROR namenode.NameNode: > java.lang.NoClassDefFoundError: org/json/JSONException > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:169) > at > org.apache.hadoop.metrics.ContextFactory.getContext(ContextFactory.java: > 132) > at > org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:56) > at > org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:45) > at > org.apache.hadoop.hdfs.server.namenode.FSDirectory.initialize(FSDirector > y.java:72) > at > org.apache.hadoop.hdfs.server.namenode.FSDirectory.(FSDirectory.ja > va:68) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.(FSNamesystem. > java:379) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:854 > ) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode. > java:948) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965) > Caused by: java.lang.ClassNotFoundException: org.json.JSONException > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > ... 11 more >=20 > 10/05/27 12:45:20 INFO namenode.NameNode: SHUTDOWN_MSG: > /************************************************************ > SHUTDOWN_MSG: Shutting down NameNode at hadoop1/10.64.147.2 > ************************************************************/ >=20 >=20 >=20 >=20 > -----Original Message----- > From: Jerome Boulon [mailto:jboulon@netflix.com] > Sent: Thursday, May 27, 2010 11:59 AM > To: chukwa-user@hadoop.apache.org > Subject: Re: Chukwa Installation Problems >=20 > Hi, > Doing ../bin/hadoop namenode -format starts a brand new java program so > that's why Chukwa gets loaded. > The java.lang.ClassNotFoundException is because you haven't put > chukwa-client-.jar into hadoop/lib directory. The > chukwa-client-.jar > should be available on all hadoop nodes. >=20 > Regards, > /Jerome. >=20 >=20 > On 5/27/10 5:14 AM, "Ratner, Alan S (IS)" wrote: >=20 >> The first steps in the Chukwa Administration Guide are to replace >> Hadoop's log4j.properties and hadoop-metrics.properties with the ones >> found in the Chukwa. Presumably this change does not take effect > until >> I restart Hadoop. But with these files in Hadoop/conf Hadoop no > longer >> works - I cannot even format the Hadoop's namenode without errors. Is >> there some sequence information I am overlooking. For example, do I >> start Chukwa before I start Hadoop? >>=20 >>=20 >> ngc@hadoop1:~/hadoop-0.20.2/conf$ ../bin/hadoop namenode -format >> log4j:ERROR Could not instantiate class >>=20 > [org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppende >> r]. >> java.lang.ClassNotFoundException: >>=20 > org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppender >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:169) >> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179) >> at >>=20 > org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionCo >> nverter.java:320) >> at >>=20 > org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverte >> r.java:121) >> at >>=20 > org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator >> .java:664) >> at >>=20 > org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator >> .java:647) >> at >>=20 > org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConf >> igurator.java:568) >> at >>=20 > org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j >> ava:442) >> at >>=20 > org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j >> ava:476) >> at >>=20 > org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConver >> ter.java:471) >> at org.apache.log4j.LogManager.(LogManager.java:125) >> at org.apache.log4j.Logger.getLogger(Logger.java:105) >> at >>=20 > org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:2 >> 29) >> at >>=20 > org.apache.commons.logging.impl.Log4JLogger.(Log4JLogger.java:65) >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> at >>=20 > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA >> ccessorImpl.java:39) >> at >>=20 > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons >> tructorAccessorImpl.java:27) >> at >> java.lang.reflect.Constructor.newInstance(Constructor.java:513) >> at >>=20 > org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImp >> l.java:529) >> at >>=20 > org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp >> l.java:235) >> at >>=20 > org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp >> l.java:209) >> at >> org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351) >> at >> org.apache.hadoop.conf.Configuration.(Configuration.java:139) >> at >>=20 > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:1 >> 01) >> log4j:ERROR Could not instantiate appender named "MR_CLIENTTRACE". >> log4j:ERROR Could not instantiate class >>=20 > [org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppende >> r]. >> java.lang.ClassNotFoundException: >>=20 > org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppender >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:169) >> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179) >> at >>=20 > org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionCo >> nverter.java:320) >> at >>=20 > org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverte >> r.java:121) >> at >>=20 > org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator >> .java:664) >> at >>=20 > org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator >> .java:647) >> at >>=20 > org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConf >> igurator.java:568) >> at >>=20 > org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j >> ava:442) >> at >>=20 > org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j >> ava:476) >> at >>=20 > org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConver >> ter.java:471) >> at org.apache.log4j.LogManager.(LogManager.java:125) >> at org.apache.log4j.Logger.getLogger(Logger.java:105) >> at >>=20 > org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:2 >> 29) >> at >>=20 > org.apache.commons.logging.impl.Log4JLogger.(Log4JLogger.java:65) >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> at >>=20 > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA >> ccessorImpl.java:39) >> at >>=20 > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons >> tructorAccessorImpl.java:27) >> at >> java.lang.reflect.Constructor.newInstance(Constructor.java:513) >> at >>=20 > org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImp >> l.java:529) >> at >>=20 > org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp >> l.java:235) >> at >>=20 > org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp >> l.java:209) >> at >> org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351) >> at >> org.apache.hadoop.conf.Configuration.(Configuration.java:139) >> at >>=20 > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:1 >> 01) >> log4j:ERROR Could not instantiate appender named "DRFAAUDIT". >> log4j:ERROR Could not instantiate class >>=20 > [org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppende >> r]. >> java.lang.ClassNotFoundException: >>=20 > org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppender >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:169) >> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179) >> at >>=20 > org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionCo >> nverter.java:320) >> at >>=20 > org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverte >> r.java:121) >> at >>=20 > org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator >> .java:664) >> at >>=20 > org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator >> .java:647) >> at >>=20 > org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConf >> igurator.java:568) >> at >>=20 > org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j >> ava:442) >> at >>=20 > org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j >> ava:476) >> at >>=20 > org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConver >> ter.java:471) >> at org.apache.log4j.LogManager.(LogManager.java:125) >> at org.apache.log4j.Logger.getLogger(Logger.java:105) >> at >>=20 > org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:2 >> 29) >> at >>=20 > org.apache.commons.logging.impl.Log4JLogger.(Log4JLogger.java:65) >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> at >>=20 > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA >> ccessorImpl.java:39) >> at >>=20 > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons >> tructorAccessorImpl.java:27) >> at >> java.lang.reflect.Constructor.newInstance(Constructor.java:513) >> at >>=20 > org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImp >> l.java:529) >> at >>=20 > org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp >> l.java:235) >> at >>=20 > org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp >> l.java:209) >> at >> org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351) >> at >> org.apache.hadoop.conf.Configuration.(Configuration.java:139) >> at >>=20 > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:1 >> 01) >> log4j:ERROR Could not instantiate appender named "HDFS_CLIENTTRACE". >> 10/05/27 08:00:08 INFO namenode.NameNode: STARTUP_MSG: >> /************************************************************ >> STARTUP_MSG: Starting NameNode >> STARTUP_MSG: host =3D hadoop1/10.64.147.2 >> STARTUP_MSG: args =3D [-format] >> STARTUP_MSG: version =3D 0.20.2 >> STARTUP_MSG: build =3D >> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r >> 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010 >> ************************************************************/ >> Re-format filesystem in /tmp/hadoop-ngc/dfs/name ? (Y or N) Y >> 10/05/27 08:01:05 INFO namenode.FSNamesystem: >> fsOwner=3Dngc,ngc,adm,dialout,cdrom,plugdev,admin,lpadmin,sambashare >> 10/05/27 08:01:05 INFO namenode.FSNamesystem: supergroup=3Dsupergroup >> 10/05/27 08:01:05 INFO namenode.FSNamesystem: isPermissionEnabled=3Dtrue >> 10/05/27 08:01:05 ERROR metrics.MetricsUtil: Unable to create metrics >> context dfs >> java.lang.ClassNotFoundException: >> org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:169) >> at >>=20 > org.apache.hadoop.metrics.ContextFactory.getContext(ContextFactory.java: >> 132) >> at >> org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:56) >> at >> org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:45) >> at >>=20 > org.apache.hadoop.hdfs.server.namenode.FSDirectory.initialize(FSDirector >> y.java:72) >> at >>=20 > org.apache.hadoop.hdfs.server.namenode.FSDirectory.(FSDirectory.ja >> va:68) >> at >>=20 > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.(FSNamesystem. >> java:379) >> at >>=20 > org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:854 >> ) >> at >>=20 > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode. >> java:948) >> at >>=20 > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965) >> 10/05/27 08:01:05 INFO common.Storage: Image file of size 93 saved in > 0 >> seconds. >> 10/05/27 08:01:05 INFO common.Storage: Storage directory >> /tmp/hadoop-ngc/dfs/name has been successfully formatted. >> 10/05/27 08:01:05 INFO namenode.NameNode: SHUTDOWN_MSG: >> /************************************************************ >> SHUTDOWN_MSG: Shutting down NameNode at hadoop1/10.64.147.2 >> ************************************************************/ >>=20 >=20 >=20