hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suhail Rehman <suhailreh...@gmail.com>
Subject Using SequenceFiles in Hadoop for an imaging application.
Date Tue, 19 Jan 2010 17:15:55 GMT
I am using hadoop to write some sample code which takes every image and
blurs it using a blurring filter.

I was able to convert all my input images into a sequence file, and I've
written the following hadoop code to perform the blurring operation. (The
input sequencefile key-value pairs are Text (filename of the image),
BytesWritable (image contents) for each record).

've used the TAR to sequence file creator available here:
http://stuartsierra.com/2008/04/24/a-million-little-files

But for some reason, I cannot get it working. I've pasted the code here (not
including the imports), let me know what I'm doing wrong (I'm new to Hadoop
btw).

import com.jhlabs.image.BoxBlurFilter;

//Mapper Class

public class BlurMapper extends MapReduceBase implements
Mapper<Text,BytesWritable,Text,BytesWritable> {

        public void map(Text key, BytesWritable file,
                        OutputCollector<Text, BytesWritable> output,
Reporter reporter) throws
IOException {


                   //Read Current Image from File.
                   BufferedImage img = ImageIO.read(new ByteArrayInputStream
(file.getBytes()));
                   BufferedImage dest = null;

                   //Apply Blur on Filter Operation - External JAR
                   BoxBlurFilter BlurOp = new BoxBlurFilter(10,10,2);
                   BlurOp.filter(img, dest);

                   ByteArrayOutputStream outputbytes = new
ByteArrayOutputStream();
                   ImageIO.write(dest, "jpeg", outputbytes);
                   BytesWritable outfile = new
BytesWritable(outputbytes.toByteArray());

                   output.collect(key, outfile);
        }

}

//MAIN CLASS

public class BlurVideoHadoop {

        public static void main(String[] args) {

                if(args.length!=2) {
                        System.err.println("Usage: blurvideo input output");
                        System.exit(-1);

                }
                JobClient client = new JobClient();
                JobConf conf = new JobConf(BlurVideoHadoop.class);

                conf.setOutputKeyClass(Text.class);
                conf.setOutputValueClass(BytesWritable.class);

                SequenceFileInputFormat.addInputPath(conf, new Path(args[0]));
                SequenceFileOutputFormat.setOutputPath(conf, new Path(args[1]));


                conf.setMapperClass(BlurMapper.class);



conf.setReducerClass(org.apache.hadoop.mapred.lib.IdentityReducer.class);

                client.setConf(conf);
                try {
                        JobClient.runJob(conf);
                } catch (Exception e) {
                        e.printStackTrace();
                }
        }



Thanks,

Regards,

Suhail Rehman
MS by Research in Computer Science
International Institute of Information Technology - Hyderabad
rehman@research.iiit.ac.in
---------------------------------------------------------------------
http://research.iiit.ac.in/~rehman

Mime
View raw message