hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Smith <ryan.justin.sm...@gmail.com>
Subject Re: Using Configuration instead of JobConf
Date Wed, 21 Oct 2009 20:12:10 GMT
Also, FileInputFormat is deprecated in hadoop 0.20.1.

On Wed, Oct 21, 2009 at 10:15 AM, Mark Vigeant <mark.vigeant@riskmetrics.com
> wrote:

> Hi Oliver,
>
> I ran into the same problem a few weeks ago. What you want to do now is use
> the classes Job and Configuration. Here's basically what you need to do:
>
>
> Configuration conf = new Configuration(); //creates the configuration
> object
> conf.set(TableOutputFormat.OUTPUT_TABLE, name); //set the output format:
> I'm using this for HBase so it's an HTable.
> Job job = new Job(conf, NAME);
> job.setJarByClass(TableUploader.class);
> job.setInputFormatClass(FileInputFormat.class);
> job.setMapperClass(MapExample.class);
> job.setNumReduceTasks(0); //alternately, you could set a Reducer class.
> Check the API
> FileInputFormat.setInputPaths(job, new Path(input));
> job.setOutputFormatClass(TableOutputFormat.class);
> job.waitForCompletion(true); //submits the job, waits for it to be
> completed.
>
> -----Original Message-----
> From: Oliver B. Fischer [mailto:o.b.fischer@swe-blog.net]
> Sent: Wednesday, October 21, 2009 8:49 AM
> To: common-user@hadoop.apache.org
> Subject: Using Configuration instead of JobConf
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> according to the API-Dokumentation of 0.20.1 JobConf is deprecated and
> we should use Configuration instead. However all examples on the webpage
>  still referece JobConf.
>
> Is there a good example for replacing JobConf by Configuration?
>
> Regards, Oliver
>
> - --
> Oliver B. Fischer, Schönhauser Allee 64, 10437 Berlin
> Tel. +49 30 44793251, Mobil: +49 178 7903538
> Mail: o.b.fischer@swe-blog.net Blog: http://www.swe-blog.net
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQEcBAEBAgAGBQJK3wNFAAoJELeiwuwqd1DGf4YH/3P83TQh+7L0GiKT4rdIe1HC
> LdZ1weVcqnBKfFZYCsL64PfwPGhkl0hoTU/sU29qGGrnAN8QKbF/8m2P32RlysCV
> ONT2nUlugHfEb8iQzF8Xx3uuCzU3AXYHK0eqme+thzfscxZKiGKzLTYb26qFnufG
> AtTIEKMkv5SRLIn5YRFM/49dcSp2WIRCIEcc+0eofsxFvI9X2a8Sg8AMG5JzNqYT
> SHMGSr3boBc4gFCNTCsRmJQIusPVWyJZICVss0q3491xdMVvGfAPTsbb6BLINFkL
> j5BrbC9OaNnmX3X3kR2/orABwMj9y5Xd1YjXepeWG3QD3WftPVyoCIOE1v8m2s0=
> =OFkK
> -----END PGP SIGNATURE-----
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message