pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-4130) Store/Load the same file fails for AvroStorage/OrcStorage, etc
Date Wed, 01 Oct 2014 23:56:33 GMT

     [ https://issues.apache.org/jira/browse/PIG-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Dai updated PIG-4130:
----------------------------
    Fix Version/s:     (was: 0.14.0)
                   0.15.0

> Store/Load the same file fails for AvroStorage/OrcStorage, etc
> --------------------------------------------------------------
>
>                 Key: PIG-4130
>                 URL: https://issues.apache.org/jira/browse/PIG-4130
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>            Priority: Minor
>             Fix For: 0.15.0
>
>
> The following script fail:
> {code}
> a = load '/user/pig/tests/data/singlefile/studenttab10k' as (name:chararray, age:int,
gpa:float);
> store a into 'Avro.intermediate' using OrcStorage();
> b = load 'Avro.intermediate' using OrcStorage();
> c = filter b by age < 30;
> store c into 'ooo';
> {code}
> Message:
> <file Avro_1.pig, line 7, column 16> Invalid field projection. Projected field
\[age\] does not exist.
> If put a "exec" after the first store, the script success.
> Pig does compile the script into two MR job, and correctly figure out the dependency
of the two, but it still need to goes for "Avro.intermediate" for the schema of b when compiling,
and at this time "Avro.intermediate" does not exist. This also happens to other Loaders which
need to get the schema from input file, such as OrcStorage, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message