hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pantazis Deligiannis <akis1...@gmail.com>
Subject Re: Compiling a mapreduce job under the new Apache Hadoop YARN
Date Sat, 11 Aug 2012 16:16:45 GMT
Worked perfectly!

Thanks a lot! :)

On 11 Aug 2012, at 16:54, Harsh J <harsh@cloudera.com> wrote:

> Hi Pantazis,
> It is better to use maven or other such tools to develop MR Java
> programs, as that handles dependencies for you. In maven you may use
> the hadoop-client POM.
> Alternatively, if you have a hadoop setup available, do this:
> $ javac -cp `hadoop classpath`:. -d class_dir Example.java
> The command "hadoop classpath" will add all requisite jars from the
> installed version to the java(c) classpath automatically.
> On Sat, Aug 11, 2012 at 7:25 AM, Pantazis Deligiannis
> <akis1986@gmail.com> wrote:
>> In the old Hadoop mapreduce v1 framework (e.g. 1.0.3) the way to compile a job was
using the already provided "hadoop-core-HADOOP_VERSION.jar" through the following command:
>>    javac -cp ".../hadoop-core-HADOOP_VERSION.jar" -d class_dir Example.java
>> In the newest releases for Hadoop mapreduce v2 YARN (e.g. 2.0.1-alpha or the cloudera
4.0) a similar core.jar is not provided, though, to be used during compilation. The closest
i have managed to get is by using the "hadoop-common-2.0.1-alpha.jar" in the /share/hadoop/common
dir but this still leaves a lot of compilation errors.
>> Is there some other specific way or using different jars for compiling a YARN job?
Or maybe for some reason you need to build the source yourself in the new releases to get
the core.jar? I have searched quite a lot but can't find a relevant answer.
>> Thank you very much!
> -- 
> Harsh J

View raw message