hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shengkai Zhu" <geniusj...@gmail.com>
Subject Re: JobClient question
Date Fri, 11 Jul 2008 15:41:52 GMT
You should provide JobTracker address and port through configuration.


On 7/11/08, Larry Compton <lawrence.compton@gmail.com> wrote:
>
> I'm coming up to speed on the Hadoop APIs. I need to be able to invoke a
> job
> from within a Java application (as opposed to running from the command-line
> "hadoop" executable). The JobConf and JobClient appear to support this and
> I've written a test program to configure and run a job. However, the job
> doesn't appear to be submitted to the JobTracker. Here's a code excerpt
> from
> my client...
>
>        String rdfInputPath = args[0];
>        String outputPath = args[1];
>        String uriInputPath = args[2];
>        String jarPath = args[3];
>
>        JobConf conf = new JobConf(MaterializeMap.class);
>        conf.setJobName("materialize");
>
>        conf.setOutputKeyClass(Text.class);
>        conf.setOutputValueClass(Text.class);
>
>        conf.setMapperClass(MaterializeMapper.class);
>        conf.setCombinerClass(MaterializeReducer.class);
>        conf.setReducerClass(MaterializeReducer.class);
>        conf.setJar(jarPath);
>
>        DistributedCache.addCacheFile(new Path(uriInputPath).toUri(), conf);
>
>        FileInputFormat.setInputPaths(conf, new Path(rdfInputPath));
>        FileOutputFormat.setOutputPath(conf, new Path(outputPath));
>
>        conf.setInputFormat(TextInputFormat.class);
>        conf.setOutputFormat(TextOutputFormat.class);
>
>        JobClient.runJob(conf);
>
> It seems like I should be providing a URL to the JobTracker somewhere, but
> I
> can't figure out where to provide the information.
>
> --
> Larry Compton
>



-- 

朱盛凯

Jash Zhu

复旦大学软件学院

Software School, Fudan University
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message