hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jingkei Ly (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3378) Incorrect sort order if WritableComparable.compareTo() does not return -1, 0 or 1
Date Tue, 13 May 2008 10:30:55 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jingkei Ly updated HADOOP-3378:
-------------------------------

    Description: 
I've found that incorrect sort orders are returned in some cases if the WritableComparable.compareTo()
method doesn't return  -1, 0 or 1. 

I believe this is a bug as the compareTo() interface states that the returned int be only
a -ve or +ve number, and will potentially catch a lot of people out who decide to write a
WritableComparator (well it caught me out anyway!). 

I'll attach an example application to demonstrate -- I simply modified the sort example to
specify a non-default comparator to sort LongWritable , i.e.:

public int compare(WritableComparable a, WritableComparable b)	{
  LongWritable longA =(LongWritable) a;
  LongWritable longB =(LongWritable) b;

  return (int) (longA.get() - longB.get());  // wrong sort order
  // return (int) Math.signum(longA.get() - longB.get());  // correct sort order
}


When I run the application through Hadoop on my input data it returns the incorrect sort order.

  was:
I've found that incorrect sort orders are returned in some cases if the WritableComparable.compareTo()
method doesn't return  -1, 0 or 1. 

I believe this is a bug as the compareTo() interface states that the returned int be only
a -ve or +ve number, and will potentially catch a lot of people out who decide to write a
WritableComparator (well it caught me out anyway!). 

I'll attach an example application to demonstrate -- I simply modified the sort example to
specify a non-default comparator to sort LongWritable , i.e.:

<pre>
public int compare(WritableComparable a, WritableComparable b)	{
	LongWritable longA =(LongWritable) a;
	LongWritable longB =(LongWritable) b;
	
	return (int) (longA.get() - longB.get());  // wrong sort order
	// return (int) Math.signum(longA.get() - longB.get());  // correct sort order
}
</pre>

When I run the application through Hadoop on my input data it returns the incorrect sort order.


> Incorrect sort order if WritableComparable.compareTo() does not return -1, 0 or 1
> ---------------------------------------------------------------------------------
>
>                 Key: HADOOP-3378
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3378
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 0.16.3
>            Reporter: Jingkei Ly
>            Priority: Minor
>         Attachments: hadoop-bug.csv, Sort.java
>
>
> I've found that incorrect sort orders are returned in some cases if the WritableComparable.compareTo()
method doesn't return  -1, 0 or 1. 
> I believe this is a bug as the compareTo() interface states that the returned int be
only a -ve or +ve number, and will potentially catch a lot of people out who decide to write
a WritableComparator (well it caught me out anyway!). 
> I'll attach an example application to demonstrate -- I simply modified the sort example
to specify a non-default comparator to sort LongWritable , i.e.:
> public int compare(WritableComparable a, WritableComparable b)	{
>   LongWritable longA =(LongWritable) a;
>   LongWritable longB =(LongWritable) b;
>   return (int) (longA.get() - longB.get());  // wrong sort order
>   // return (int) Math.signum(longA.get() - longB.get());  // correct sort order
> }
> When I run the application through Hadoop on my input data it returns the incorrect sort
order.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message