pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-4773) [Pig on Tez] Secondary key descending sort in nested foreach after union does ascending instead
Date Mon, 11 Jan 2016 06:12:39 GMT

    [ https://issues.apache.org/jira/browse/PIG-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15091488#comment-15091488
] 

Rohini Palaniswamy commented on PIG-4773:
-----------------------------------------

bq. Do we need to set "pig.secondarySortOrder" again at vertex level? We already set it at
edge level.
  Main reason for setting it at vertex level is for comparator intialization in POShuffleTezLoad.
POShuffleTezLoad makes use of comparator for key comparison for multiple inputs. Even though
we don't have the case of multiple inputs (cogroup) with SecondaryKeyComparison now which
is the code path in which it will be exercised, added it now itself to avoid missing that
in future and added a TODO. 



> [Pig on Tez] Secondary key descending sort in nested foreach after union does ascending
instead
> -----------------------------------------------------------------------------------------------
>
>                 Key: PIG-4773
>                 URL: https://issues.apache.org/jira/browse/PIG-4773
>             Project: Pig
>          Issue Type: Bug
>          Components: tez
>            Reporter: Rohini Palaniswamy
>            Assignee: Rohini Palaniswamy
>             Fix For: 0.16.0
>
>         Attachments: PIG-4773-1.patch, PIG-4773-2.patch
>
>
>   PigSecondaryKeyComparator does not implement a compare(WritableComparable a, WritableComparable
b) method. In case of  OrderedGroupedMergedKVInput with vertex groups, object comparison instead
of byte comparison is done when merging the multiple inputs into one. Since the compare API
is not overridden, WritableComparator compare API is called and  the value types are directly
compared which means it is always ascending.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message