pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec Cecho (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-3633) AvroStorage tests are failing when running against Avro 1.7.5
Date Wed, 18 Dec 2013 09:34:09 GMT

     [ https://issues.apache.org/jira/browse/PIG-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jarek Jarcec Cecho updated PIG-3633:
------------------------------------

    Attachment: PIG-3633.patch

Attaching patch that will skip schema parsing if the given string is empty. This seems to
resolve the failing tests.

> AvroStorage tests are failing when running against Avro 1.7.5
> -------------------------------------------------------------
>
>                 Key: PIG-3633
>                 URL: https://issues.apache.org/jira/browse/PIG-3633
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.12.0
>            Reporter: Jarek Jarcec Cecho
>            Assignee: Jarek Jarcec Cecho
>            Priority: Minor
>             Fix For: 0.13.0
>
>         Attachments: PIG-3633.patch
>
>
> {{AvroStorage}} tests executed against latest Avro release 1.7.5 are failing:
> {code}
> ant clean test -Dtestcase=TestAvroStorage -Davro.version=1.7.5
> ...
>    [junit] Running org.apache.pig.builtin.TestAvroStorage
>    [junit] Tests run: 33, Failures: 15, Errors: 0, Time elapsed: 14.211 sec
> {code}
> With following exception:
> {code}
> Failed to parse: Pig script failed to parse: 
> <line 1, column 5> pig script failed to validate: java.lang.RuntimeException: could
not instantiate 'AvroStorage' with arguments '[, -r]'
> 	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196)
> 	at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676)
> 	at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1623)
> 	at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
> 	at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
> 	at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
> 	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
> 	at org.apache.pig.PigServer.registerScript(PigServer.java:649)
> 	at org.apache.pig.PigServer.registerScript(PigServer.java:726)
> 	at org.apache.pig.PigServer.registerScript(PigServer.java:699)
> 	at org.apache.pig.builtin.TestAvroStorage.testAvroStorage(TestAvroStorage.java:775)
> 	at org.apache.pig.builtin.TestAvroStorage.testLoadRecursiveRecordsOptionOn(TestAvroStorage.java:588)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> 	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38)
> 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
> 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
> 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
> Caused by: 
> <line 1, column 5> pig script failed to validate: java.lang.RuntimeException: could
not instantiate 'AvroStorage' with arguments '[, -r]'
> 	at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:849)
> 	at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3479)
> 	at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1536)
> 	at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013)
> 	at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553)
> 	at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
> 	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
> 	... 34 more
> Caused by: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments
'[, -r]'
> 	at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:748)
> 	at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:837)
> 	... 40 more
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> 	at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:716)
> 	... 41 more
> Caused by: java.lang.NullPointerException
> 	at org.apache.avro.Schema.parse(Schema.java:1072)
> 	at org.apache.avro.Schema$Parser.parse(Schema.java:950)
> 	at org.apache.avro.Schema$Parser.parse(Schema.java:940)
> 	at org.apache.pig.builtin.AvroStorage.<init>(AvroStorage.java:136)
> 	... 46 more
> 3701 [main] WARN  org.apache.pig.tools.parameters.PreprocessorContext  - Warning : Multiple
values found for INFILE. Using value test/org/apache/pig/builtin/avro/data/trevni/uncompressed/simpleRecordsTrevni.trevni
> 3701 [main] WARN  org.apache.pig.tools.parameters.PreprocessorContext  - Warning : Multiple
values found for AVROSTORAGE_OUT_2. Using value -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc
> 3702 [main] WARN  org.apache.pig.tools.parameters.PreprocessorContext  - Warning : Multiple
values found for OUTFILE. Using value /home/jarcec/cloudera/repos/pig/build/test/TestAvroStorage/testLoadTrevniRecords
> 3732 [main] ERROR org.apache.pig.PigServer  - exception during parsing: Error during
parsing. Pig script failed to parse: 
> <line 4, column 0> pig script failed to validate: java.lang.RuntimeException: could
not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]'
> Failed to parse: Pig script failed to parse: 
> <line 4, column 0> pig script failed to validate: java.lang.RuntimeException: could
not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]'
> 	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196)
> 	at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676)
> 	at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1623)
> 	at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
> 	at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
> 	at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
> 	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
> 	at org.apache.pig.PigServer.registerScript(PigServer.java:649)
> 	at org.apache.pig.PigServer.registerScript(PigServer.java:726)
> 	at org.apache.pig.PigServer.registerScript(PigServer.java:699)
> 	at org.apache.pig.builtin.TestAvroStorage.testAvroStorage(TestAvroStorage.java:775)
> 	at org.apache.pig.builtin.TestAvroStorage.testLoadTrevniRecords(TestAvroStorage.java:709)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> 	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38)
> 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
> 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
> 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
> Caused by: 
> <line 4, column 0> pig script failed to validate: java.lang.RuntimeException: could
not instantiate 'AvroStorage' with arguments '[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]'
> 	at org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:936)
> 	at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7691)
> 	at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1580)
> 	at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013)
> 	at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553)
> 	at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
> 	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
> 	... 34 more
> Caused by: java.lang.RuntimeException: could not instantiate 'AvroStorage' with arguments
'[, -f test/org/apache/pig/builtin/avro/schema/simpleRecordsTrevni.avsc]'
> 	at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:748)
> 	at org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:915)
> 	... 40 more
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> 	at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:716)
> 	... 41 more
> Caused by: java.lang.NullPointerException
> 	at org.apache.avro.Schema.parse(Schema.java:1072)
> 	at org.apache.avro.Schema$Parser.parse(Schema.java:950)
> 	at org.apache.avro.Schema$Parser.parse(Schema.java:940)
> 	at org.apache.pig.builtin.AvroStorage.<init>(AvroStorage.java:136)
> 	... 46 more
> {code}
> [~tomwhite] has looked into the problem and found out that Avro  in version 1.7.5 has
updated the JSON parser library Jackson from 1.8.8 to 1.9.13. The new version is changing
the way empty strings are processed.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message