avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pierre Mariani (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1266) Fix mapred AvroMultipleOutputs class to write the schema to Jobconf rather than private Hashmap
Date Tue, 05 Mar 2013 17:24:14 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13593615#comment-13593615
] 

Pierre Mariani commented on AVRO-1266:
--------------------------------------

I am getting a null pointer exception when using avro 1.7.4 and the patch from March 3rd.

Details of the exception:
{noformat}
java.lang.NullPointerException
        at java.io.StringReader.<init>(StringReader.java:33)
        at org.apache.avro.Schema$Parser.parse(Schema.java:917)
        at org.apache.avro.Schema.parse(Schema.java:966)
        at org.apache.avro.mapred.AvroMultipleOutputs$InternalFileOutputFormat.getRecordWriter(AvroMultipleOutputs.java:611)
        at org.apache.avro.mapred.AvroMultipleOutputs.getRecordWriter(AvroMultipleOutputs.java:411)
        at org.apache.avro.mapred.AvroMultipleOutputs.getCollector(AvroMultipleOutputs.java:570)
        at org.apache.avro.mapred.AvroMultipleOutputs.getCollector(AvroMultipleOutputs.java:506)
        <call to amos.getCollector("outA", reporter).collect(object);>
{noformat}

Configuration of my job:
{noformat}
JobConf conf = new JobConf(getConf(), getClass());
AvroJob.setInputSchema(conf, schemaA);
AvroJob.setMapOutputSchema(conf, Pair.getPairSchema(Schema.create(Schema.Type.STRING), schemaA));
AvroJob.setOutputSchema(conf, schemaA);
AvroJob.setMapperClass(conf, MyMapper.class);
AvroJob.setReducerClass(conf, MyReducer.class);
AvroMultipleOutputs.addNamedOutput(conf, "outA", AvroOutputFormat.class, schemaA);
AvroMultipleOutputs.addNamedOutput(conf, "outB", AvroOutputFormat.class, schemaB);
{noformat}
                
> Fix mapred AvroMultipleOutputs class to write the schema to Jobconf rather than private
Hashmap
> -----------------------------------------------------------------------------------------------
>
>                 Key: AVRO-1266
>                 URL: https://issues.apache.org/jira/browse/AVRO-1266
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>            Reporter: Ashish Nagavaram
>            Assignee: Ashish Nagavaram
>             Fix For: 1.7.5
>
>         Attachments: AVRO-1266.patch, AVRO-1266.patch
>
>
> The current version of mapred AvroMultipleOutputs stores schemas in provate hashmap which
has issues when run in a mapreduce code. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message