pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PIG-2689) JsonStorage fails to find schema when LimitAdjuster runs
Date Sun, 13 Jul 2014 01:17:05 GMT

    [ https://issues.apache.org/jira/browse/PIG-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059971#comment-14059971
] 

Rohini Palaniswamy edited comment on PIG-2689 at 7/13/14 1:15 AM:
------------------------------------------------------------------

The issue is already fixed by PIG-3120. But that does not have a unit test. So added a unit
test as part of this patch and also set the alias on the new limit operator based on [~ddaniels888]
original patch. The unit test on the original patch does not apply cleanly anymore as having
files for test is removed from TestJsonStorage and they are generated at run time.


was (Author: rohini):
The issue is already fixed by PIG-3120. But that does not have a unit test. So added a unit
test as part of this patch and also set the alias on the new limit operator.

> JsonStorage fails to find schema when LimitAdjuster runs
> --------------------------------------------------------
>
>                 Key: PIG-2689
>                 URL: https://issues.apache.org/jira/browse/PIG-2689
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.10.0
>            Reporter: Doug Daniels
>            Assignee: Doug Daniels
>             Fix For: 0.14.0
>
>         Attachments: PIG-2689-2-nowhitespacechange.patch, PIG-2689-2.patch, PIG-2689.patch
>
>
> Scripts that both save out data with JsonStorage and trigger the LimitAdjuster (e.g.
doing an order by followed by a limit) yield the following Exception:
> java.io.IOException: Could not find schema in UDF context
>         at org.apache.pig.builtin.JsonStorage.prepareToWrite(JsonStorage.java:125)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86)
>         at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:569)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:638)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
>         at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260)
> This happens b/c the LimitAdjuster does not copy the signature into it's newly created
POStore, and hence JsonStorage looks for the schema for a null signature.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message