hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raymond Jennings III <raymondj...@yahoo.com>
Subject Re: Cutom partitioner question
Date Thu, 03 Jun 2010 19:22:08 GMT
Hi Ted, that does not appear to be the problem I am having.  I tried adding it as you said
but I get the same runtime error.  Here is my partitioner:

  public class MyPartitioner extends Partitioner<Text, Text> {
	  
	public MyPartitioner() {
		
	}
	  
	public int getPartition(Text key, Text value, int num_partitions) {
		String key2 = key.toString();
		int hash = key2.hashCode();
		
		hash = hash % num_partitions;
	
		return(hash);
	}	  
  }


and in my main I have:

job.setMapOutputValueClass(Text.class);
job.setMapOutputKeyClass(Text.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);

job.setPartitionerClass(MyPartitioner.class);


Thanks.

--- On Thu, 6/3/10, Ted Yu <yuzhihong@gmail.com> wrote:

> From: Ted Yu <yuzhihong@gmail.com>
> Subject: Re: Cutom partitioner question
> To: common-user@hadoop.apache.org
> Date: Thursday, June 3, 2010, 2:10 PM
> An empty ctor is needed for your
> Partitioner class.
> 
> On Thu, Jun 3, 2010 at 10:13 AM, Raymond Jennings III
> <raymondjiii@yahoo.com
> > wrote:
> 
> > I am trying to create my partitioner but I am getting
> an exception.  Is
> > anything required other than providing the method
> "public int getPartition"
> > and extending the Partitioner class?
> >
> >
> >
> > java.lang.RuntimeException:
> java.lang.NoSuchMethodException:
> > TSPmrV6$TSPPartitioner.<init>()
> >        at
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> >        at
> >
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:527)
> >        at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> >        at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> >        at
> org.apache.hadoop.mapred.Child.main(Child.java:170)
> > Caused by: java.lang.NoSuchMethodException:
> TSPmrV6$TSPPartitioner.<init>()
> >        at
> java.lang.Class.getConstructor0(Unknown Source)
> >        at
> java.lang.Class.getDeclaredConstructor(Unknown Source)
> >        at
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
> >        ... 4 more
> >
> >
> >
> >
> >
> 


      

Mime
View raw message