Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of paul.van.hoven@googlemail.com
 designates 74.125.83.51 as permitted sender)
MIME-Version: 1.0
Date: Tue, 18 Dec 2012 17:18:58 +0100
Message-ID: 
 <CANqzD4PzQPd-exM2h+=iohvw_4YKrkF-8V99TW1G1qLvUpWXcw@mail.gmail.com>
Subject: Job does not finish due to java.lang.ClassCastException: class
 java.util.Date
From: Paul van Hoven <paul.van.hoven@googlemail.com>
To: user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hi,

I wrote a small java program for processing some log files (line after
line separated by newline). I'm using ecm on amazon to perform my job.
I wrote the following code:

public static class Map extends Mapper<LongWritable, Text, Date, Object> {

		private LineParser lineParser = new LineParser();

		public void map( LongWritable key, Text value, Context context )
throws IOException, InterruptedException {

			Click click = lineParser.parseClick( value.toString() );
			if( click != null ) {
				context.write( click.timestamp, click );
				return;
			}
			
			Conversion conversion = lineParser.parseConversion( value.toString() );
			if( conversion != null ) {
				context.write( conversion.timestamp, conversion );
				return;
			}

		}
	}


	public static class Reduce extends Reducer<Date, Object, Text, Text> {

		public void reduce( Date key, Iterable<Object> values, Context
context) throws IOException, InterruptedException {

			SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss Z");
			for( Object obj : values ) {
				if( obj.getClass().getName().equals("myHadoopProject.Click") )
					context.write( new Text( sdf.format( key ) ), new Text( ((Click)
obj).toHadoopString() ) );
				else
					context.write( new Text( sdf.format( key ) ), new Text( ((Click)
obj).toHadoopString() ) );
			}
			
		}

	}


	public static void main(String[] args) throws Exception {
		Configuration conf = new Configuration();

		Job job = new Job(conf, "MyHadoopJob");
		job.setJarByClass(MyHadoopClass.class);

		job.setOutputKeyClass(Date.class);
		job.setOutputValueClass(Object.class);

		job.setMapperClass(Map.class);
		job.setReducerClass(Reduce.class);

		job.setInputFormatClass(TextInputFormat.class);
		job.setOutputFormatClass(TextOutputFormat.class);

		FileInputFormat.addInputPath( job, new Path( args[0] ) );
		FileOutputFormat.setOutputPath( job, new Path( args[1] ) );

		job.waitForCompletion(true);
	}


Unfortunately I get the following Exception and the job fails:


java.lang.ClassCastException: class java.util.Date
	at java.lang.Class.asSubclass(Class.java:3018)
	at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:786)
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:975)
	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:681)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)


What is wrong with my code?