Mailing-List: contact dev-help@pig.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@pig.apache.org
Date: Mon, 16 Feb 2015 10:01:12 +0000 (UTC)
From: "Zoltan Talas (JIRA)" <jira@apache.org>
To: pig-dev@hadoop.apache.org
Message-ID: <JIRA.12775322.1424080869000.54102.1424080872780@Atlassian.JIRA>
In-Reply-To: <JIRA.12775322.1424080869000@Atlassian.JIRA>
References: <JIRA.12775322.1424080869000@Atlassian.JIRA>
 <JIRA.12775322.1424080869994@arcas>
Subject: [jira] [Created] (PIG-4423) AvroStorage() does not validate schema
 at storing.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Zoltan Talas created PIG-4423:
---------------------------------

             Summary: AvroStorage() does not validate schema at storing.
                 Key: PIG-4423
                 URL: https://issues.apache.org/jira/browse/PIG-4423
             Project: Pig
          Issue Type: Bug
          Components: piggybank
    Affects Versions: 0.12.0
         Environment: EMR AMI 3.3.2
            Reporter: Zoltan Talas


Pig does not validate Avro schema when using AvroStorage(). I tried to validate schema both by adding schema_file input parameter and by providing schema explicitly as well. Both cases Avro file received the schema of Pig data set instead of validating schema from Avro file. When i have used the same Avro schema for Hive, it validated data successfully (if data has different schema compared to Avro then threw an error)

store data into '$TARGET'
USING AvroStorage(
'schema', '{
"type": "record",
"name": "test",
"fields": [
{"name": "partner_name", "type": "string"},
{"name": "partner_id", "type": "int"},
{"name": "name", "type": "string"} ,
{"name": "id", "type": "int"}
]
}');

or

STORE data INTO '$TARGET' 
USING AvroStorage('schema_file','$AVRO_SCHEMA');

I have registered the following jars (downloaded from Maven repo)

REGISTER piggybank-0.12.0.jar;
REGISTER avro-1.7.7.jar;
REGISTER avro-mapred-1.7.7.jar;
REGISTER json-simple-1.1.1.jar;


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)