Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=mime-version:in-reply-to:references:content-type:message-id:
	content-transfer-encoding:from:subject:date:to:x-mailer;
	b=FTeEon+5ACrgsvDy7G0gcjF0bQ+nZeY+Sc33EoTCFxIFgLob7UeU/b+snZ5ph6BW
Mime-Version: 1.0 (Apple Message framework v753)
In-Reply-To: <1b29507a0804141625s15a9784eh4350c7116e803cd2@mail.gmail.com>
References: <1b29507a0804141625s15a9784eh4350c7116e803cd2@mail.gmail.com>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <6BC2F9CE-0BFE-4B80-BF5D-09DFC360B917@yahoo-inc.com>
Content-Transfer-Encoding: 7bit
From: arkady borkovsky <arkady@yahoo-inc.com>
Subject: Re: Pre-sort value list in reduce
Date: Tue, 15 Apr 2008 00:32:30 -0700
To: core-dev@hadoop.apache.org

look at
  -partitioner org.apache.hadoop.mapred.lib.KeyFieldBasedPartitioner

--ab

On Apr 14, 2008, at 4:25 PM, pi song wrote:

> Dear people in Hadoop mailing list,
>
> Is there any way to control the value list in reduce (Key, List of  
> values)
> to be sorted? or at least clusteringly sorted (containing clusters  
> of sorted
> values e.g. 1,1,1,2,2,2,2,3,3,3,  1,1,1,1,1,1,2,2,2,2,3
> ,1,1,2,2,2,3,3,3,3,3,3,3) ?
> I had a look at JobConf.setOutputValueGroupingComparator in javadoc  
> and I
> think it might be the answer because I feel most of the time  
> grouping in
> Hadoop is done by sort. Am I right?
>
> Can anyone help me? How about the performance impact of your solution?
>
> Thanks in advance,
> Pi