incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jerome Boulon <jbou...@netflix.com>
Subject Re: Chukwa Installation Problems
Date Thu, 27 May 2010 17:02:09 GMT
My bad,
You also need json.jar in hadoop/lib/ directory.
/Jerome


On 5/27/10 9:58 AM, "Ratner, Alan S (IS)" <Alan.Ratner@ngc.com> wrote:

> Thanks for the tip.  This detail seems to have been omitted from
> http://www.cs.berkeley.edu/~asrabkin/chukwa/admin.html
> and http://wiki.apache.org/hadoop/Chukwa_Console_Integration_Guide.  So
> I copied chukwa-0.4.0/chukwa-hadoop-0.4.0-client.jar (the closest name
> match I could get for chukwa-client-<xx>.jar) to all the hadoop/lib
> directories in my cluster and copied log4j.properties and
> hadoop-metrics.properties to all the hadoop/conf directories in my
> cluster.  But now when I reformat the namenode I get a different error:
> 
> ngc@hadoop1:~/hadoop-0.20.2$ bin/hadoop namenode -format
> 10/05/27 12:45:17 INFO namenode.NameNode: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting NameNode
> STARTUP_MSG:   host = hadoop1/10.64.147.2
> STARTUP_MSG:   args = [-format]
> STARTUP_MSG:   version = 0.20.2
> STARTUP_MSG:   build =
> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r
> 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
> ************************************************************/
> Re-format filesystem in /tmp/hadoop-ngc/dfs/name ? (Y or N) Y
> 10/05/27 12:45:20 INFO namenode.FSNamesystem:
> fsOwner=ngc,ngc,adm,dialout,cdrom,plugdev,admin,lpadmin,sambashare
> 10/05/27 12:45:20 INFO namenode.FSNamesystem: supergroup=supergroup
> 10/05/27 12:45:20 INFO namenode.FSNamesystem: isPermissionEnabled=true
> 10/05/27 12:45:20 ERROR namenode.NameNode:
> java.lang.NoClassDefFoundError: org/json/JSONException
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at
> org.apache.hadoop.metrics.ContextFactory.getContext(ContextFactory.java:
> 132)
> at
> org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:56)
> at
> org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:45)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.initialize(FSDirector
> y.java:72)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.<init>(FSDirectory.ja
> va:68)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.
> java:379)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:854
> )
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.
> java:948)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
> Caused by: java.lang.ClassNotFoundException: org.json.JSONException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> ... 11 more
> 
> 10/05/27 12:45:20 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at hadoop1/10.64.147.2
> ************************************************************/
> 
> 
> 
> 
> -----Original Message-----
> From: Jerome Boulon [mailto:jboulon@netflix.com]
> Sent: Thursday, May 27, 2010 11:59 AM
> To: chukwa-user@hadoop.apache.org
> Subject: Re: Chukwa Installation Problems
> 
> Hi,
> Doing ../bin/hadoop namenode -format starts a brand new java program so
> that's why Chukwa gets loaded.
> The java.lang.ClassNotFoundException is because you haven't put
> chukwa-client-<xx>.jar into hadoop/lib directory. The
> chukwa-client-<xx>.jar
> should be available on all hadoop nodes.
> 
> Regards,
>   /Jerome.
> 
> 
> On 5/27/10 5:14 AM, "Ratner, Alan S (IS)" <Alan.Ratner@ngc.com> wrote:
> 
>> The first steps in the Chukwa Administration Guide are to replace
>> Hadoop's log4j.properties and hadoop-metrics.properties with the ones
>> found in the Chukwa.  Presumably this change does not take effect
> until
>> I restart Hadoop.  But with these files in Hadoop/conf Hadoop no
> longer
>> works - I cannot even format the Hadoop's namenode without errors.  Is
>> there some sequence information I am overlooking.  For example, do I
>> start Chukwa before I start Hadoop?
>> 
>> 
>> ngc@hadoop1:~/hadoop-0.20.2/conf$ ../bin/hadoop namenode -format
>> log4j:ERROR Could not instantiate class
>> 
> [org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppende
>> r].
>> java.lang.ClassNotFoundException:
>> 
> org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppender
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:169)
>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
>> at
>> 
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionCo
>> nverter.java:320)
>> at
>> 
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverte
>> r.java:121)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator
>> .java:664)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator
>> .java:647)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConf
>> igurator.java:568)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j
>> ava:442)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j
>> ava:476)
>> at
>> 
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConver
>> ter.java:471)
>> at org.apache.log4j.LogManager.<clinit>(LogManager.java:125)
>> at org.apache.log4j.Logger.getLogger(Logger.java:105)
>> at
>> 
> org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:2
>> 29)
>> at
>> 
> org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>> at
>> 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
>> ccessorImpl.java:39)
>> at
>> 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
>> tructorAccessorImpl.java:27)
>> at
>> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>> at
>> 
> org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImp
>> l.java:529)
>> at
>> 
> org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp
>> l.java:235)
>> at
>> 
> org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp
>> l.java:209)
>> at
>> org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351)
>> at
>> org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:139)
>> at
>> 
> org.apache.hadoop.hdfs.server.namenode.NameNode.<clinit>(NameNode.java:1
>> 01)
>> log4j:ERROR Could not instantiate appender named "MR_CLIENTTRACE".
>> log4j:ERROR Could not instantiate class
>> 
> [org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppende
>> r].
>> java.lang.ClassNotFoundException:
>> 
> org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppender
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:169)
>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
>> at
>> 
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionCo
>> nverter.java:320)
>> at
>> 
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverte
>> r.java:121)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator
>> .java:664)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator
>> .java:647)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConf
>> igurator.java:568)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j
>> ava:442)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j
>> ava:476)
>> at
>> 
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConver
>> ter.java:471)
>> at org.apache.log4j.LogManager.<clinit>(LogManager.java:125)
>> at org.apache.log4j.Logger.getLogger(Logger.java:105)
>> at
>> 
> org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:2
>> 29)
>> at
>> 
> org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>> at
>> 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
>> ccessorImpl.java:39)
>> at
>> 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
>> tructorAccessorImpl.java:27)
>> at
>> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>> at
>> 
> org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImp
>> l.java:529)
>> at
>> 
> org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp
>> l.java:235)
>> at
>> 
> org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp
>> l.java:209)
>> at
>> org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351)
>> at
>> org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:139)
>> at
>> 
> org.apache.hadoop.hdfs.server.namenode.NameNode.<clinit>(NameNode.java:1
>> 01)
>> log4j:ERROR Could not instantiate appender named "DRFAAUDIT".
>> log4j:ERROR Could not instantiate class
>> 
> [org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppende
>> r].
>> java.lang.ClassNotFoundException:
>> 
> org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppender
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:169)
>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
>> at
>> 
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionCo
>> nverter.java:320)
>> at
>> 
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverte
>> r.java:121)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator
>> .java:664)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator
>> .java:647)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConf
>> igurator.java:568)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j
>> ava:442)
>> at
>> 
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.j
>> ava:476)
>> at
>> 
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConver
>> ter.java:471)
>> at org.apache.log4j.LogManager.<clinit>(LogManager.java:125)
>> at org.apache.log4j.Logger.getLogger(Logger.java:105)
>> at
>> 
> org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:2
>> 29)
>> at
>> 
> org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>> at
>> 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
>> ccessorImpl.java:39)
>> at
>> 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
>> tructorAccessorImpl.java:27)
>> at
>> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>> at
>> 
> org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImp
>> l.java:529)
>> at
>> 
> org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp
>> l.java:235)
>> at
>> 
> org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImp
>> l.java:209)
>> at
>> org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351)
>> at
>> org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:139)
>> at
>> 
> org.apache.hadoop.hdfs.server.namenode.NameNode.<clinit>(NameNode.java:1
>> 01)
>> log4j:ERROR Could not instantiate appender named "HDFS_CLIENTTRACE".
>> 10/05/27 08:00:08 INFO namenode.NameNode: STARTUP_MSG:
>> /************************************************************
>> STARTUP_MSG: Starting NameNode
>> STARTUP_MSG:   host = hadoop1/10.64.147.2
>> STARTUP_MSG:   args = [-format]
>> STARTUP_MSG:   version = 0.20.2
>> STARTUP_MSG:   build =
>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r
>> 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
>> ************************************************************/
>> Re-format filesystem in /tmp/hadoop-ngc/dfs/name ? (Y or N) Y
>> 10/05/27 08:01:05 INFO namenode.FSNamesystem:
>> fsOwner=ngc,ngc,adm,dialout,cdrom,plugdev,admin,lpadmin,sambashare
>> 10/05/27 08:01:05 INFO namenode.FSNamesystem: supergroup=supergroup
>> 10/05/27 08:01:05 INFO namenode.FSNamesystem: isPermissionEnabled=true
>> 10/05/27 08:01:05 ERROR metrics.MetricsUtil: Unable to create metrics
>> context dfs
>> java.lang.ClassNotFoundException:
>> org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:169)
>> at
>> 
> org.apache.hadoop.metrics.ContextFactory.getContext(ContextFactory.java:
>> 132)
>> at
>> org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:56)
>> at
>> org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:45)
>> at
>> 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.initialize(FSDirector
>> y.java:72)
>> at
>> 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.<init>(FSDirectory.ja
>> va:68)
>> at
>> 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.
>> java:379)
>> at
>> 
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:854
>> )
>> at
>> 
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.
>> java:948)
>> at
>> 
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
>> 10/05/27 08:01:05 INFO common.Storage: Image file of size 93 saved in
> 0
>> seconds.
>> 10/05/27 08:01:05 INFO common.Storage: Storage directory
>> /tmp/hadoop-ngc/dfs/name has been successfully formatted.
>> 10/05/27 08:01:05 INFO namenode.NameNode: SHUTDOWN_MSG:
>> /************************************************************
>> SHUTDOWN_MSG: Shutting down NameNode at hadoop1/10.64.147.2
>> ************************************************************/
>> 
> 
> 


Mime
View raw message