pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-4298) Descending order-by is broken in some cases when key is bytearrays
Date Fri, 07 Nov 2014 21:57:34 GMT

     [ https://issues.apache.org/jira/browse/PIG-4298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Cheolsoo Park updated PIG-4298:
-------------------------------
       Resolution: Fixed
    Fix Version/s:     (was: 0.15.0)
                   0.14.0
           Status: Resolved  (was: Patch Available)

Committed to 0.14 and trunk.

> Descending order-by is broken in some cases when key is bytearrays 
> -------------------------------------------------------------------
>
>                 Key: PIG-4298
>                 URL: https://issues.apache.org/jira/browse/PIG-4298
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
>             Fix For: 0.14.0
>
>         Attachments: PIG-4298-1.patch, PIG-4298-2.patch, repo.tar.gz
>
>
> Here is a repo script (using [PigPen|https://github.com/Netflix/PigPen] )-
> {code}
> REGISTER pigpen.jar;
> load4254 = LOAD 'input.clj'
>     USING PigStorage('\n')
>     AS (value:chararray);
> DEFINE udf4265 pigpen.PigPenFnDataBag('(clojure.core/require (quote [pigpen.runtime])
(quote [clojure.edn]))','(pigpen.runtime/exec [(pigpen.runtime/process->bind (pigpen.runtime/pre-process
:pig :native)) (pigpen.runtime/map->bind clojure.edn/read-string) (pigpen.runtime/key-selector->bind
clojure.core/identity) (pigpen.runtime/process->bind (pigpen.runtime/post-process :pig
:native-key-frozen-val))])');
> generate4263 = FOREACH load4254 GENERATE
>     FLATTEN(udf4265(value));
> generate4257 = FOREACH generate4263 GENERATE
>     $0 AS key,
>     $1 AS value;
> order4258 = ORDER generate4257 BY key DESC; <-- sort order isn't changed by DESC
> dump order4258;
> {code}
> This script returns the same result for both ASC and DESC orders.
> The problem is as follows-
> # {{PigBytesRawComparator}} calls {{BinInterSedesTupleRawComparator.compare()}}.
> # {{BinInterSedesTupleRawComparator}} applies descending order.
> # {{PigBytesRawComparator}} applies descending order again to what {{BinInterSedesTupleRawComparator}}
returns.
> Therefore, descending order is never applied.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message