pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Gates <ga...@yahoo-inc.com>
Subject Re: Proposal for adding user defined ordering to pig
Date Tue, 06 Nov 2007 21:10:34 GMT
"You can ... register it in WritableComparator".  Does this mean call 
WritableComparator.define(), passing the user provided Comparator 
implementation?  Is that what you're proposing Ben?  
JobConf.setOutputKeyComparatorClass() requires a WritableComparator, 
which means the user would need to subclass that rather than just 
implement the Comparator interface.


Benjamin Reed wrote:
> Exactly. Just to be clear. We are talking about how to implement the ordering. 
> Pig developers would just need to implement the Comparator class as outlined 
> by Alan.
> ben
> On Tuesday 06 November 2007 03:53:20 Andrzej Bialecki wrote:
>> Benjamin Reed wrote:
>>> Using the compareTo method is just the default. We can define our own
>>> WritetableComparator subclass to hook into the Hadoop sorting.
>> You can either register it in WritableComparator (though this is less
>> flexible - the comparator will be used in every case, no matter what job
>> runs in the JVM), or set it specifically on a per-job basis using
>> JobConf.setOutputKeyComparatorClass().

View raw message