Return-Path: 
 <user-return-2220-apmail-avro-user-archive=avro.apache.org@avro.apache.org>
X-Original-To: apmail-avro-user-archive@www.apache.org
Delivered-To: apmail-avro-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 70DE1DAF3
	for <apmail-avro-user-archive@www.apache.org>;
 Mon,  4 Mar 2013 07:50:49 +0000 (UTC)
Received: (qmail 9901 invoked by uid 500); 4 Mar 2013 07:50:49 -0000
Delivered-To: apmail-avro-user-archive@avro.apache.org
Received: (qmail 9361 invoked by uid 500); 4 Mar 2013 07:50:44 -0000
Mailing-List: contact user-help@avro.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@avro.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@avro.apache.org>
List-Post: <mailto:user@avro.apache.org>
List-Id: <user.avro.apache.org>
Reply-To: user@avro.apache.org
Delivered-To: mailing list user@avro.apache.org
Received: (qmail 9335 invoked by uid 99); 4 Mar 2013 07:50:44 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Mar 2013 07:50:44 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of esammer@cloudera.com
 designates 209.85.160.50 as permitted sender)
Received: from [209.85.160.50] (HELO mail-pb0-f50.google.com) (209.85.160.50)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Mar 2013 07:50:39 +0000
Received: by mail-pb0-f50.google.com with SMTP id up1so2942888pbc.37
        for <user@avro.apache.org>; Sun, 03 Mar 2013 23:50:19 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20120113;
        h=mime-version:x-received:date:message-id:subject:from:to
         :content-type:x-gm-message-state;
        bh=meD/3JSPB4K6ba8X2jg2EwzBQIkCSX1/MHozafii6/A=;
        b=ngYut26Puni6ZifSX9raBmLG7GX2CpKBY/D1qZTyVsh8a+p6fAqqCdMj1otvq+WLT7
         ORhZfbxeqXGc2uz/UTtOYtxPpJw4nkhZhhLfQMit1GZ+OLZu8qHBAiwJPIJW3JusFc2O
         CFUYr0j89k+35LMqNrD1666KLeevoyToyUAPj/wAqpG3s63DCpoKFkjlfq7MQQ4ZZwQM
         BtokD89CR1u7qgDZWGkb9iZRAroEQxVOY7NoJ8gtxLfchbyPD9hYuvRlZF3kuwsQ+MI7
         H1mACAaqKutS21eQSpX8Tav0PVB7W1hsub3zNKMOGk0oJQjRHumybtRL62dcMBmvXQR6
         obrw==
MIME-Version: 1.0
X-Received: by 10.66.85.161 with SMTP id i1mr31515226paz.67.1362383419407;
 Sun, 03 Mar 2013 23:50:19 -0800 (PST)
Received: by 10.68.42.161 with HTTP; Sun, 3 Mar 2013 23:50:19 -0800 (PST)
Date: Sun, 3 Mar 2013 23:50:19 -0800
Message-ID: 
 <CANpnispd+zPvPZEiSAyEtbMmKfa=tKaF2uukQF9B+p1+nje5Eg@mail.gmail.com>
Subject: Anonymous record schemas in data files
From: Eric Sammer <esammer@cloudera.com>
To: user@avro.apache.org
Content-Type: multipart/alternative; boundary=f46d042ef5a5c3187404d7149bcb
X-Gm-Message-State: 
 ALoCoQnbicA47OqpfEjbztyAUO2xTMhzrKGXPe6mtkOPd9LxUu/k32jiqux+KpezPd4dKKS6qEXr
X-Virus-Checked: Checked by ClamAV on apache.org

--f46d042ef5a5c3187404d7149bcb
Content-Type: text/plain; charset=ISO-8859-1

All:

I'm looking for some clarity on the use of anonymous records in Avro data
files. Is this considered legal? 1.7.3 allows one to write a data file with
DataFileWriter with an anonymous record schema that can't be read back
which is not the nicest behavior. Here's a contrived example of a data file:

esammer:~/ esammer$ ~/bin/avro-tool getmeta 1362381940987-1
Exception in thread "main" org.apache.avro.SchemaParseException: No name in
schema: {"type":"record","fields":[{"name":"word","type":"string"}]}
        at org.apache.avro.Schema.getRequiredText(Schema.java:1198)
        at org.apache.avro.Schema.parse(Schema.java:1066)
        at org.apache.avro.Schema$Parser.parse(Schema.java:927)
        at org.apache.avro.Schema$Parser.parse(Schema.java:917)
        at org.apache.avro.Schema.parse(Schema.java:974)
        at
org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:124)
        at
org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97)
        at
org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:89)
        at
org.apache.avro.tool.DataFileGetMetaTool.run(DataFileGetMetaTool.java:63)
        at org.apache.avro.tool.Main.run(Main.java:78)
        at org.apache.avro.tool.Main.main(Main.java:67)

Before I filed the bug I wanted to clarify that anonymous records are
against the spec (or that they aren't, and the bug is the schema parser).

Thanks.
-- 
Eric Sammer
twitter: esammer
data: www.cloudera.com

--f46d042ef5a5c3187404d7149bcb
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">All:<div><br></div><div>I&#39;m looking for some clarity o=
n the use of anonymous records in Avro data files. Is this considered legal=
? 1.7.3 allows one to write a data file with DataFileWriter with an anonymo=
us record schema that can&#39;t be read back which is not the nicest behavi=
or. Here&#39;s a contrived example of a data file:</div>
<div><br></div><div><div>esammer:~/ esammer$ ~/bin/avro-tool getmeta 136238=
1940987-1=A0</div><div>Exception in thread &quot;main&quot; org.apache.avro=
.SchemaParseException: No name in schema: {&quot;type&quot;:&quot;record&qu=
ot;,&quot;fields&quot;:[{&quot;name&quot;:&quot;word&quot;,&quot;type&quot;=
:&quot;string&quot;}]}</div>
<div>=A0 =A0 =A0 =A0 at org.apache.avro.Schema.getRequiredText(Schema.java:=
1198)</div><div>=A0 =A0 =A0 =A0 at org.apache.avro.Schema.parse(Schema.java=
:1066)</div><div>=A0 =A0 =A0 =A0 at org.apache.avro.Schema$Parser.parse(Sch=
ema.java:927)</div><div>
=A0 =A0 =A0 =A0 at org.apache.avro.Schema$Parser.parse(Schema.java:917)</di=
v><div>=A0 =A0 =A0 =A0 at org.apache.avro.Schema.parse(Schema.java:974)</di=
v><div>=A0 =A0 =A0 =A0 at org.apache.avro.file.DataFileStream.initialize(Da=
taFileStream.java:124)</div>
<div>=A0 =A0 =A0 =A0 at org.apache.avro.file.DataFileReader.&lt;init&gt;(Da=
taFileReader.java:97)</div><div>=A0 =A0 =A0 =A0 at org.apache.avro.file.Dat=
aFileReader.&lt;init&gt;(DataFileReader.java:89)</div><div>=A0 =A0 =A0 =A0 =
at org.apache.avro.tool.DataFileGetMetaTool.run(DataFileGetMetaTool.java:63=
)</div>
<div>=A0 =A0 =A0 =A0 at org.apache.avro.tool.Main.run(Main.java:78)</div><d=
iv>=A0 =A0 =A0 =A0 at org.apache.avro.tool.Main.main(Main.java:67)</div><di=
v><br></div><div style>Before I filed the bug I wanted to clarify that anon=
ymous records are against the spec (or that they aren&#39;t, and the bug is=
 the schema parser).</div>
<div style><br></div><div style>Thanks.</div>-- <br>Eric Sammer<br>twitter:=
 esammer<br>data: <a href=3D"http://www.cloudera.com">www.cloudera.com</a>
</div></div>

--f46d042ef5a5c3187404d7149bcb--