hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Young Kim <juneng...@gmail.com>
Subject Re: is there more smarter way to execute a hadoop cluster?
Date Thu, 24 Feb 2011 09:06:27 GMT
hello, harsh.

to use MultipleOutput class,
I need to use a Job class to set it as a first argument to configure 
about my hadoop job.

|*addNamedOutput 
<http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html#addNamedOutput%28org.apache.hadoop.mapreduce.Job,%20java.lang.String,%20java.lang.Class,%20java.lang.Class,%20java.lang.Class%29>*(Job

<http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/Job.html>
job,String 
<http://java.sun.com/javase/6/docs/api/java/lang/String.html?is-external=true> namedOutput,Class

<http://java.sun.com/javase/6/docs/api/java/lang/Class.html?is-external=true><? 
extendsOutputFormat 
<http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/OutputFormat.html>>
outputFormatClass,Class 
<http://java.sun.com/javase/6/docs/api/java/lang/Class.html?is-external=true><?>
keyClass,Class 
<http://java.sun.com/javase/6/docs/api/java/lang/Class.html?is-external=true><?>
valueClass)|
           Adds a named output for the job.

AYK, Job class is deprecated in 0.21.0.

to submit my job in a cluster like runJob().

How are I going to do?

Junyoung Kim (juneng603@gmail.com)


On 02/24/2011 04:12 PM, Harsh J wrote:
> Hello,
>
> On Thu, Feb 24, 2011 at 12:25 PM, Jun Young Kim<juneng603@gmail.com>  wrote:
>> Hi,
>> I executed my cluster by this way.
>>
>> call a command in shell directly.
> What are you doing within your testCluster.jar? If you are simply
> submitting a job, you can use a Driver method and get rid of all these
> hassles. JobClient and Job classes both support submitting jobs from
> Java API itself.
>
> Please read the tutorial on submitting application code via code
> itself: http://developer.yahoo.com/hadoop/tutorial/module4.html#driver
> Notice the last line in the code presented there, which submits a job
> itself. Using runJob() also prints your progress/counters etc.
>
> The way you've implemented this looks unnecessary when your Jar itself
> can be made runnable with a Driver!
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message