pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nandor Kollar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-5167) Limit_4 is failing with spark exec type
Date Fri, 10 Mar 2017 09:47:04 GMT

    [ https://issues.apache.org/jira/browse/PIG-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904794#comment-15904794
] 

Nandor Kollar commented on PIG-5167:
------------------------------------

[~xuefuz] in Pig, as far as I know, we compare the results after sorting each time.
[~knoguchi] sortArgs doesn't help, AFAIK those parameters are used to check if the output
is sorted or not. These sort arguments tell the key to check for.
I'm wondering why this is not a problem on MR/Tez mode, [~rohini] do you have any idea why
this test is not failing in MR?

> Limit_4 is failing with spark exec type
> ---------------------------------------
>
>                 Key: PIG-5167
>                 URL: https://issues.apache.org/jira/browse/PIG-5167
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: Nandor Kollar
>            Assignee: Nandor Kollar
>             Fix For: spark-branch
>
>         Attachments: PIG-5167.patch
>
>
> results are different:
> {code}
> diff <(head -n 5 Limit_4.out/out_sorted) <(head -n 5 Limit_4_benchmark.out/out_sorted)
> 1,5c1,5
> < 	50	3.00
> < 	74	2.22
> < alice carson	66	2.42
> < alice quirinius	71	0.03
> < alice van buren	28	2.50
> ---
> > bob allen		0.28
> > bob allen	22	0.92
> > bob allen	25	2.54
> > bob allen	26	2.35
> > bob allen	27	2.17
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message