Return-Path: X-Original-To: apmail-avro-user-archive@www.apache.org Delivered-To: apmail-avro-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AAB93D908 for ; Tue, 5 Mar 2013 16:38:14 +0000 (UTC) Received: (qmail 38291 invoked by uid 500); 5 Mar 2013 16:38:14 -0000 Delivered-To: apmail-avro-user-archive@avro.apache.org Received: (qmail 36526 invoked by uid 500); 5 Mar 2013 16:38:02 -0000 Mailing-List: contact user-help@avro.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@avro.apache.org Delivered-To: mailing list user@avro.apache.org Received: (qmail 35701 invoked by uid 99); 5 Mar 2013 16:37:59 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Mar 2013 16:37:59 +0000 Received: from localhost (HELO mail-vb0-f41.google.com) (127.0.0.1) (smtp-auth username cutting, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Mar 2013 16:37:59 +0000 Received: by mail-vb0-f41.google.com with SMTP id l22so1357938vbn.28 for ; Tue, 05 Mar 2013 08:37:58 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.52.70.33 with SMTP id j1mr8432932vdu.23.1362501478249; Tue, 05 Mar 2013 08:37:58 -0800 (PST) Received: by 10.58.18.163 with HTTP; Tue, 5 Mar 2013 08:37:57 -0800 (PST) In-Reply-To: <51351B62.8040404@gmail.com> References: <51351B62.8040404@gmail.com> Date: Tue, 5 Mar 2013 08:37:57 -0800 Message-ID: Subject: Re: different output schema when using org.apache.avro.mapred.AvroMultipleOutputs? From: Doug Cutting To: user@avro.apache.org Content-Type: text/plain; charset=UTF-8 Can you please add this comment to that issue in Jira? Thanks! Doug On Mon, Mar 4, 2013 at 2:08 PM, Pierre Mariani wrote: > It looks like what I am looking for, but after applying the patch to avro > mapred 1.7.4 I get the following exception: > > java.lang.NullPointerException > at java.io.StringReader.(StringReader.java:33) > at org.apache.avro.Schema$Parser.parse(Schema.java:917) > at org.apache.avro.Schema.parse(Schema.java:966) > at > org.apache.avro.mapred.AvroMultipleOutputs$InternalFileOutputFormat.getRecordWriter(AvroMultipleOutputs.java:611) > at > org.apache.avro.mapred.AvroMultipleOutputs.getRecordWriter(AvroMultipleOutputs.java:411) > at > org.apache.avro.mapred.AvroMultipleOutputs.getCollector(AvroMultipleOutputs.java:570) > at > org.apache.avro.mapred.AvroMultipleOutputs.getCollector(AvroMultipleOutputs.java:506) > at > com.inome.diffhadoop.DiffSets$DiffReducer.reduce(DiffSets.java:93) > at > com.inome.diffhadoop.DiffSets$DiffReducer.reduce(DiffSets.java:40) > at > org.apache.avro.mapred.HadoopReducerBase.reduce(HadoopReducerBase.java:60) > at > org.apache.avro.mapred.HadoopReducerBase.reduce(HadoopReducerBase.java:30) > at > org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:463) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > > > On 03/01/2013 11:07 AM, Doug Cutting wrote: >> >> Is this the issue addressed by AVRO-1266? >> >> https://issues.apache.org/jira/browse/AVRO-1266 >> >> Doug >> >> On Thu, Feb 28, 2013 at 3:11 PM, Pierre Mariani >> wrote: >>> >>> Beside fixing an issue with baseOutputPath, >>> https://issues.apache.org/jira/browse/AVRO-1215 allows to specify >>> different >>> schema for different outputs when using AvroMultipleOutputs. >>> >>> However, this is for the new Hadoop API (org.apache.avro.mapreduce). >>> >>> Is it possible that the same bug exists in the package for the old Hadoop >>> API (org.apache.avro.mapred) ? >>> >>> Thank you >>> Pierre > >