hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mich Talebzadeh" <m...@peridale.co.uk>
Subject RE: Hadoop imports Fail
Date Tue, 14 Apr 2015 09:48:30 GMT
Great probably the easiest option is to create CLASSPATH as below and just do

 

javac –cp $CLASSPATH –d ..

 

HTH

 

Mich Talebzadeh

 

http://talebzadehmich.wordpress.com

 

Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.


co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4

Publications due shortly:

Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and Coherence Cache

Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly

 

NOTE: The information in this email is proprietary and confidential. This message is for the
designated recipient only, if you are not the intended recipient, you should destroy it immediately.
Any information in this message shall not be understood as given or endorsed by Peridale Ltd,
its subsidiaries or their employees, unless expressly so stated. It is the responsibility
of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd,
its subsidiaries nor their employees accept any responsibility.

 

From: Anand Murali [mailto:anand_vihar@yahoo.com] 
Sent: 14 April 2015 10:43
To: user@hadoop.apache.org
Subject: Re: Hadoop imports Fail

 

Many thanks. Ii fixed the problem using $javac -classpath $(hadoop classpath) -d <source
files>

 

Thanks

 

 

Anand Murali  

11/7, 'Anand Vihar', Kandasamy St, Mylapore

Chennai - 600 004, India

Ph: (044)- 28474593/ 43526162 (voicemail)

 

 

On Tuesday, April 14, 2015 2:56 PM, Mich Talebzadeh <mich@peridale.co.uk> wrote:

 

You need to create an environment file like environment.ksh or use ./kshrc in your home directory
to build env variables for hadoop etc something like below

 

export HADOOP_HOME=~/hadoop/hadoop-2.6.0

PATH="$HADOOP_HOME/bin:$HADOOP_HOME/sbin:/usr/java/latest/bin":$PATH

export PATH=~/pig-0.14.0/bin:$PATH

#HADOOP VARIABLES START

export JAVA_HOME=/usr/java/latest

export HADOOP_INSTALL=/home/hduser/hadoop/hadoop-2.6.0

export PATH=$PATH:$HADOOP_INSTALL/bin

export PATH=$PATH:$HADOOP_INSTALL/sbin

export HADOOP_MAPRED_HOME=$HADOOP_INSTALL

export HADOOP_COMMON_HOME=$HADOOP_INSTALL

export HADOOP_HDFS_HOME=$HADOOP_INSTALL

export YARN_HOME=$HADOOP_INSTALL

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native

export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"

export HADOOP_CLIENT_OPTS="-Xmx2g"

#HADOOP VARIABLES END

CLASSPATH=.:$HADOOP_HOME/share/hadoop/common/hadoop-common-2.6.0-tests.jar:$HADOOP_HOME/share/hadoop/common/hadoop-common-2.6.0.jar:hadoop-nfs-2.6.0.jar:$HIVE_HOME/conf

export CLASSPATH

 

 

HTH

 

Mich Talebzadeh

 

http://talebzadehmich.wordpress.com

 

Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.


co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4

Publications due shortly:

Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and Coherence Cache

Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly

 

NOTE: The information in this email is proprietary and confidential. This message is for the
designated recipient only, if you are not the intended recipient, you should destroy it immediately.
Any information in this message shall not be understood as given or endorsed by Peridale Ltd,
its subsidiaries or their employees, unless expressly so stated. It is the responsibility
of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd,
its subsidiaries nor their employees accept any responsibility.

 

 

-----Original Message-----
From: Andre Kelpe [mailto:akelpe@concurrentinc.com] 
Sent: 14 April 2015 09:52
To: user@hadoop.apache.org
Subject: Re: Hadoop imports Fail

 

Please just use a build tool like maven or gradle for your build.

There is no way to manage your classpath like this and stay sane.

Nobody does this and you shouldn't either.

 

- André

 

On Tue, Apr 14, 2015 at 7:34 AM, Anand Murali < <mailto:anand_vihar@yahoo.com> anand_vihar@yahoo.com>
wrote:

> Dear Naik:

>  

> I have already set path both for Hadoop and Java in .hadoop and run it 

> prior to compilation

>  

> Export JAVA_HOME=/home/anand_vihar/jdk1.7.0_75

> Export HADOOP_INSTALL=/home/anand_vihar/hadoop-2.6.0

> PATH=:$PATH:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin:$HADOOP_INSTALL/s

> hare:$JAVA_HOME

>  

>  

> So, is there not a way that the compiler goes thru all folders/jar and 

> pick up the right class and copile with just

>  

> $javac *.java.

>  

>  

> If not is there a way to do so, else every time compilation needs to 

> be done then a complete path list has to be typed and who knows which 

> class is located in which jar which is needed for compilation, which 

> when not found gives a compile error which becomes difficult to debug.

>  

> Thanks

>  

> Regards

>  

> Anand Murali

> 11/7, 'Anand Vihar', Kandasamy St, Mylapore Chennai - 600 004, India

> Ph: (044)- 28474593/ 43526162 (voicemail)

>  

>  

>  

> On Monday, April 13, 2015 9:58 PM, Ravindra Kumar Naik

> < <mailto:ravin.iitb@gmail.com> ravin.iitb@gmail.com> wrote:

>  

>  

> You need to set classpath to hadoop jar

>  

> java -classpath <path/to/hadoop-2.6-core.jar>" MaxTemperatureMapper.java

>  

>  

>  

> On Mon, Apr 13, 2015 at 11:33 AM, Anand Murali < <mailto:anand_vihar@yahoo.com>
anand_vihar@yahoo.com>

> wrote:

>  

> Dear Mr. Ted:

>  

> Please find details

>  

> Last login: Sun Apr 12 16:33:18 2015 from localhost

> anand_vihar@Latitude-E5540:~$ cd hadoop-2.6.0

>  <mailto:anand_vihar@Latitude-E5540:~/hadoop-2.6.0$> anand_vihar@Latitude-E5540:~/hadoop-2.6.0$
. .hadoop

>  <mailto:anand_vihar@Latitude-E5540:~/hadoop-2.6.0$> anand_vihar@Latitude-E5540:~/hadoop-2.6.0$
cd etc/hadoop

>  <mailto:anand_vihar@Latitude-E5540:~/hadoop-2.6.0/etc/hadoop$> anand_vihar@Latitude-E5540:~/hadoop-2.6.0/etc/hadoop$
sh hadoop-env.sh

>  <mailto:anand_vihar@Latitude-E5540:~/hadoop-2.6.0/etc/hadoop$> anand_vihar@Latitude-E5540:~/hadoop-2.6.0/etc/hadoop$
java -version

> java version "1.7.0_75"

> Java(TM) SE Runtime Environment (build 1.7.0_75-b13)

> Java HotSpot(TM) 64-Bit Server VM (build 24.75-b04, mixed mode)

>  <mailto:anand_vihar@Latitude-E5540:~/hadoop-2.6.0/etc/hadoop$> anand_vihar@Latitude-E5540:~/hadoop-2.6.0/etc/hadoop$
hadoop version

> Hadoop 2.6.0

> Subversion  <https://git-wip-us.apache.org/repos/asf/hadoop.git> https://git-wip-us.apache.org/repos/asf/hadoop.git
-r

> e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1

> Compiled by jenkins on 2014-11-13T21:10Z

> Compiled with protoc 2.5.0

> From source with checksum 18e43357c8f927c0695f1e9522859d6a

> This command was run using

> /home/anand_vihar/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar

>  <mailto:anand_vihar@Latitude-E5540:~/hadoop-2.6.0/etc/hadoop$> anand_vihar@Latitude-E5540:~/hadoop-2.6.0/etc/hadoop$
cd ..

>  <mailto:anand_vihar@Latitude-E5540:~/hadoop-2.6.0/etc$> anand_vihar@Latitude-E5540:~/hadoop-2.6.0/etc$
cd ..

>  <mailto:anand_vihar@Latitude-E5540:~/hadoop-2.6.0$> anand_vihar@Latitude-E5540:~/hadoop-2.6.0$
start-dfs.sh --config

> /home/anand_vihar/hadoop-2.6.0/conf

>  

> Error Messages

>  

>  

> $javac MaxTemperatureMapper.java

>  

> symbol: class Mapper

> MaxTemperatureMapper.java:8: error: cannot find symbol

> public class MaxTemperatureMapper extends Mapper<LongWritable, Text, Text,

> IntWritable>

>                                                  ^

>   symbol: class LongWritable

> MaxTemperatureMapper.java:8: error: cannot find symbol

> public class MaxTemperatureMapper extends Mapper<LongWritable, Text, Text,

> IntWritable>

>                                                                ^

>   symbol: class Text

> MaxTemperatureMapper.java:8: error: cannot find symbol

> public class MaxTemperatureMapper extends Mapper<LongWritable, Text, Text,

> IntWritable>

>                                                                      ^

>   symbol: class Text

> MaxTemperatureMapper.java:8: error: cannot find symbol

> public class MaxTemperatureMapper extends Mapper<LongWritable, Text, Text,

> IntWritable>

>                                                                            ^

>   symbol: class IntWritable

> MaxTemperatureMapper.java:15: error: cannot find symbol

>     public void map(LongWritable key, Text value, Context context)

>                     ^

>   symbol:   class LongWritable

>   location: class MaxTemperatureMapper

> MaxTemperatureMapper.java:15: error: cannot find symbol

>     public void map(LongWritable key, Text value, Context context)

>                                       ^

>   symbol:   class Text

>   location: class MaxTemperatureMapper

> MaxTemperatureMapper.java:15: error: cannot find symbol

>     public void map(LongWritable key, Text value, Context context)

>                                                   ^

>   symbol:   class Context

>   location: class MaxTemperatureMapper

> MaxTemperatureMapper.java:25: error: cannot find symbol

>             airtemperature = Integer.parseInt(line.substring(87,92));

>             ^

>   symbol:   variable airtemperature

>   location: class MaxTemperatureMapper

> MaxTemperatureMapper.java:28: error: cannot find symbol

>         if (airTemparature != MISSING && quality.matches("[01459]"))

>             ^

>   symbol:   variable airTemparature

>   location: class MaxTemperatureMapper

> MaxTemperatureMapper.java:30: error: cannot find symbol

>             context.write(new Text(year), new IntWritable(airTemperature));

>                               ^

>   symbol:   class Text

>   location: class MaxTemperatureMapper

> MaxTemperatureMapper.java:30: error: cannot find symbol

>             context.write(new Text(year), new IntWritable(airTemperature));

>                                               ^

>   symbol:   class IntWritable

>   location: class MaxTemperatureMapper

> MaxTemperatureMapper.java:14: error: method does not override or implement a

> method from a supertype

>     @Override

>     ^

> 17 errors

>  

> Hope someone can advise. Look forward to hearing.

>  

> Thanks

>  

> Regards

>  

> Anand Murali

> 11/7, 'Anand Vihar', Kandasamy St, Mylapore

> Chennai - 600 004, India

> Ph: (044)- 28474593/ 43526162 (voicemail)

>  

>  

>  

> On Sunday, April 12, 2015 5:59 PM, Ted Yu < <mailto:yuzhihong@gmail.com> yuzhihong@gmail.com>
wrote:

>  

>  

> Can you tell us how you setup your project?

>  

> Do you use maven to build it ?

>  

> Please pastebin snippet of your code and the error you got.

>  

> Cheers

>  

>  

>  

> On Apr 12, 2015, at 4:16 AM, Anand Murali < <mailto:anand_vihar@yahoo.com> anand_vihar@yahoo.com>
wrote:

>  

> Dear All:

>  

> I am new to hadoop2.6.0. I managed to set up and install Hadoop and Java

> 1.7. I wrote my first hadoop code with import statements of Hadoop class

> files. However when I try and compile the code, I get errors, which are

> reflective of the class files, which I suppose is not imported. I have run

> hadoop-env.sh successfully before dameon startup. Can somebody advise.

>  

> Thanks

>  

> Anand Murali

> 11/7, 'Anand Vihar', Kandasamy St, Mylapore

> Chennai - 600 004, India

> Ph: (044)- 28474593/ 43526162 (voicemail)

>  

>  

>  

>  

>  

>  

 

 

 

-- 

André Kelpe

 <mailto:andre@concurrentinc.com> andre@concurrentinc.com

 <http://concurrentinc.com/> http://concurrentinc.com

 


Mime
View raw message