hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Zhang <zjf...@gmail.com>
Subject Re: Using SequenceFiles in Hadoop for an imaging application.
Date Wed, 20 Jan 2010 02:41:18 GMT
Could you paste your exception message ?



On Wed, Jan 20, 2010 at 1:15 AM, Suhail Rehman <suhailrehman@gmail.com>wrote:

> I am using hadoop to write some sample code which takes every image and
> blurs it using a blurring filter.
>
> I was able to convert all my input images into a sequence file, and I've
> written the following hadoop code to perform the blurring operation. (The
> input sequencefile key-value pairs are Text (filename of the image),
> BytesWritable (image contents) for each record).
>
> 've used the TAR to sequence file creator available here:
> http://stuartsierra.com/2008/04/24/a-million-little-files
>
> But for some reason, I cannot get it working. I've pasted the code here
> (not including the imports), let me know what I'm doing wrong (I'm new to
> Hadoop btw).
>
> import com.jhlabs.image.BoxBlurFilter;
>
> //Mapper Class
>
> public class BlurMapper extends MapReduceBase implements
> Mapper<Text,BytesWritable,Text,BytesWritable> {
>
>         public void map(Text key, BytesWritable file,
>
>                         OutputCollector<Text, BytesWritable> output, Reporter reporter)
throws
> IOException {
>
>
>                    //Read Current Image from File.
>                    BufferedImage img = ImageIO.read(new ByteArrayInputStream
>
> (file.getBytes()));
>                    BufferedImage dest = null;
>
>                    //Apply Blur on Filter Operation - External JAR
>                    BoxBlurFilter BlurOp = new BoxBlurFilter(10,10,2);
>                    BlurOp.filter(img, dest);
>
>                    ByteArrayOutputStream outputbytes = new ByteArrayOutputStream();
>                    ImageIO.write(dest, "jpeg", outputbytes);
>                    BytesWritable outfile = new BytesWritable(outputbytes.toByteArray());
>
>                    output.collect(key, outfile);
>         }
>
> }
>
> //MAIN CLASS
>
> public class BlurVideoHadoop {
>
>         public static void main(String[] args) {
>
>                 if(args.length!=2) {
>
>                         System.err.println("Usage: blurvideo input output");
>                         System.exit(-1);
>
>                 }
>                 JobClient client = new JobClient();
>                 JobConf conf = new JobConf(BlurVideoHadoop.class);
>
>                 conf.setOutputKeyClass(Text.class);
>                 conf.setOutputValueClass(BytesWritable.class);
>
>                 SequenceFileInputFormat.addInputPath(conf, new Path(args[0]));
>                 SequenceFileOutputFormat.setOutputPath(conf, new Path(args[1]));
>
>
>                 conf.setMapperClass(BlurMapper.class);
>
>
>                 conf.setReducerClass(org.apache.hadoop.mapred.lib.IdentityReducer.class);
>
>                 client.setConf(conf);
>                 try {
>
>                         JobClient.runJob(conf);
>                 } catch (Exception e) {
>                         e.printStackTrace();
>                 }
>         }
>
>
>
> Thanks,
>
> Regards,
>
> Suhail Rehman
> MS by Research in Computer Science
> International Institute of Information Technology - Hyderabad
> rehman@research.iiit.ac.in
> ---------------------------------------------------------------------
> http://research.iiit.ac.in/~rehman <http://research.iiit.ac.in/%7Erehman>
>



-- 
Best Regards

Jeff Zhang

Mime
View raw message