hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hailong.yang1115 <hailong.yang1...@gmail.com>
Subject [help]how to stop HDFS
Date Wed, 30 Nov 2011 10:50:17 GMT
Actually I started to play with the latest release 0.23.0 on two nodes yesterday. And it was
easy to start the hdfs. However it took me a while to configure the yarn. I set the variables
HADOOP_COMMON_HOME to where you extracted the tarball and HADOOP_HDFS_HOME to the local dir
where I pointed the hdfs to. After that I could bring up yarn and run the benchmark. But I
am facing a problem that I could not see the jobs in the UI. And also when I started the historyserver,
I got the following error.

11/11/30 20:53:19 FATAL hs.JobHistoryServer: Error starting JobHistoryServer
java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.fs.Hdfs
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1179)
        at org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:142)
        at org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:233)
        at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:315)
        at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:313)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
        at org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:313)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:426)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:448)
        at org.apache.hadoop.mapreduce.v2.hs.JobHistory.init(JobHistory.java:183)
        at org.apache.hadoop.yarn.service.CompositeService.init(CompositeService.java:58)
        at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.init(JobHistoryServer.java:62)
        at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:77)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.Hdfs
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1125)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1177)
        ... 14 more
Any clue?

Hailong




***********************************************
* Hailong Yang, PhD. Candidate 
* Sino-German Joint Software Institute, 
* School of Computer Science&Engineering, Beihang University
* Phone: (86-010)82315908
* Email: hailong.yang1115@gmail.com
* Address: G413, New Main Building in Beihang University, 
*              No.37 XueYuan Road,HaiDian District, 
*              Beijing,P.R.China,100191
***********************************************

发件人: cat fa
发送时间: 2011-11-30 10:28
收件人: common-user
主题: Re: Re: [help]how to stop HDFS
In fact it's me to say sorry. I used the word "install" which was misleading.

In fact I downloaded a tar file and extracted it to /usr/bin/hadoop

Could you please tell me where to point those variables?

2011/11/30, Prashant Sharma <prashant.iiith@gmail.com>:
> I am sorry, I had no idea you have done a rpm install, my suggestion was
> based on the assumption that you have done a tar extract install where all
> three distribution have to extracted and then export variables.
> Also I have no experience with rpm based installs - so no comments about
> what went wrong in your case.
>
> Basically from the error i can say that it is not able to find the jars
> needed  on classpath which is referred by scripts through
> HADOOP_COMMON_HOME. I would say check with the access permission as in
> which user was it installed with and which user is it running with ?
>
> On Tue, Nov 29, 2011 at 10:48 PM, cat fa <boost.subscribing@gmail.com>wrote:
>
>> Thank you for your help, but I'm still a little confused.
>> Suppose I installed hadoop in /usr/bin/hadoop/ .Should I
>> point HADOOP_COMMON_HOME to /usr/bin/hadoop ? Where should I
>> point HADOOP_HDFS_HOME? Also to /usr/bin/hadoop/ ?
>>
>> 2011/11/30 Prashant Sharma <prashant.iiith@gmail.com>
>>
>> > I mean, you have to export the variables
>> >
>> > export HADOOP_CONF_DIR=/path/to/your/configdirectory.
>> >
>> > also export HADOOP_HDFS_HOME ,HADOOP_COMMON_HOME. before your run your
>> > command. I suppose this should fix the problem.
>> > -P
>> >
>> > On Tue, Nov 29, 2011 at 6:23 PM, cat fa <boost.subscribing@gmail.com>
>> > wrote:
>> >
>> > > it didn't work. It gave me the Usage information.
>> > >
>> > > 2011/11/29 hailong.yang1115 <hailong.yang1115@gmail.com>
>> > >
>> > > > Try $HADOOP_PREFIX_HOME/bin/hdfs namenode stop --config
>> > $HADOOP_CONF_DIR
>> > > > and $HADOOP_PREFIX_HOME/bin/hdfs datanode stop --config
>> > $HADOOP_CONF_DIR.
>> > > > It would stop namenode and datanode separately.
>> > > > The HADOOP_CONF_DIR is the directory where you store your
>> configuration
>> > > > files.
>> > > > Hailong
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > ***********************************************
>> > > > * Hailong Yang, PhD. Candidate
>> > > > * Sino-German Joint Software Institute,
>> > > > * School of Computer Science&Engineering, Beihang University
>> > > > * Phone: (86-010)82315908
>> > > > * Email: hailong.yang1115@gmail.com
>> > > > * Address: G413, New Main Building in Beihang University,
>> > > > *              No.37 XueYuan Road,HaiDian District,
>> > > > *              Beijing,P.R.China,100191
>> > > > ***********************************************
>> > > >
>> > > > From: cat fa
>> > > > Date: 2011-11-29 20:22
>> > > > To: common-user
>> > > > Subject: Re: [help]how to stop HDFS
>> > > > use $HADOOP_CONF or $HADOOP_CONF_DIR ? I'm using hadoop 0.23.
>> > > >
>> > > > you mean which class? the class of hadoop or of java?
>> > > >
>> > > > 2011/11/29 Prashant Sharma <prashant.iiith@gmail.com>
>> > > >
>> > > > > Try making $HADOOP_CONF point to right classpath including your
>> > > > > configuration folder.
>> > > > >
>> > > > >
>> > > > > On Tue, Nov 29, 2011 at 3:58 PM, cat fa <
>> boost.subscribing@gmail.com
>> > >
>> > > > > wrote:
>> > > > >
>> > > > > > I used the command :
>> > > > > >
>> > > > > > $HADOOP_PREFIX_HOME/bin/hdfs start namenode --config
>> > $HADOOP_CONF_DIR
>> > > > > >
>> > > > > > to sart HDFS.
>> > > > > >
>> > > > > > This command is in Hadoop document (here
>> > > > > > <
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
>> > > > > > >)
>> > > > > >
>> > > > > > However, I got errors as
>> > > > > >
>> > > > > > Exception in thread "main" java.lang.NoClassDefFoundError:start
>> > > > > >
>> > > > > > Could anyone tell me how to start and stop HDFS?
>> > > > > >
>> > > > > > By the way, how to set Gmail so that it doesn't top post
my
>> reply?
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message