pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-3661) Piggybank AvroStorage fails if used in more than one load or store statement
Date Fri, 24 Jan 2014 22:25:39 GMT

     [ https://issues.apache.org/jira/browse/PIG-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Rohini Palaniswamy updated PIG-3661:
------------------------------------

      Resolution: Fixed
    Hadoop Flags: Reviewed
          Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks Cheolsoo for the review.

> Piggybank AvroStorage fails if used in more than one load or store statement
> ----------------------------------------------------------------------------
>
>                 Key: PIG-3661
>                 URL: https://issues.apache.org/jira/browse/PIG-3661
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.11.1
>            Reporter: Rohini Palaniswamy
>            Assignee: Rohini Palaniswamy
>             Fix For: 0.13.0
>
>         Attachments: PIG-3661-1.patch, PIG-3661-2.patch
>
>
> To reproduce:
> A =load '/tmp/data' as (a1:int, a2:int, a3:int);
> B = load '/tmp/data1' as (b1:chararray, b2:chararray, b3:chararray);
> store A into '/tmp/out/a' using org.apache.pig.piggybank.storage.avro.AvroStorage();
> store B into '/tmp/out2/b' using org.apache.pig.piggybank.storage.avro.AvroStorage();
> It either fails in the map job if schema is incompatible, or B gets schema of A and B
merged leading to incorrect results.
> Reason is schema is stored and accessed from a property of UDFContext without using a
context signature.
> UDFContext context = UDFContext.getUDFContext();
>         Properties property = context.getUDFProperties(ResourceSchema.class);
>         String prevSchemaStr = property.getProperty(AVRO_OUTPUT_SCHEMA_PROPERTY);



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message