Return-Path: X-Original-To: apmail-avro-user-archive@www.apache.org Delivered-To: apmail-avro-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C76FDE903 for ; Tue, 8 Jan 2013 09:49:30 +0000 (UTC) Received: (qmail 49892 invoked by uid 500); 8 Jan 2013 09:49:30 -0000 Delivered-To: apmail-avro-user-archive@avro.apache.org Received: (qmail 49581 invoked by uid 500); 8 Jan 2013 09:49:30 -0000 Mailing-List: contact user-help@avro.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@avro.apache.org Delivered-To: mailing list user@avro.apache.org Received: (qmail 49566 invoked by uid 99); 8 Jan 2013 09:49:29 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Jan 2013 09:49:29 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of chandra.pratyush@gmail.com designates 209.85.215.54 as permitted sender) Received: from [209.85.215.54] (HELO mail-la0-f54.google.com) (209.85.215.54) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Jan 2013 09:49:21 +0000 Received: by mail-la0-f54.google.com with SMTP id fp12so216134lab.27 for ; Tue, 08 Jan 2013 01:49:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=bQ3Q5vvaHxBPxwFQ3T4Z5JSUn+vq3Sa6A/V3dxHLNMk=; b=XAkZ2xzzMbrCP1oSD6PDeXXw4yNKsUOKkhyF7V7FXlpGh++zmDK9F92pP0hY/XkN4Y cXqWxDY+LeJhiU5pitUHkRtQ/HQyW4R579SyYXFUC1mpYl0h5tzx8J38Cs3lqSwYaf+a juaLMMKMP2gOkFkvLLVn3mo04Rw08wibeyfOrtiBOjqj6IPAfU89P3qewD47Pl/BT94X jTo5eWcY003tgi6HJwjrSfUOBKxanNipKrjQgb14941QDOzATdjtJ94bjOstFYzu3GQM XDM54ZF45YwZbzf/wKjyX4RQi3w9VdKWSsCyrXEIr+BG7BMm/LCUVatIz/RGEXxSpVo4 Lutw== MIME-Version: 1.0 Received: by 10.152.45.174 with SMTP id o14mr50443653lam.12.1357638540959; Tue, 08 Jan 2013 01:49:00 -0800 (PST) Received: by 10.112.76.98 with HTTP; Tue, 8 Jan 2013 01:49:00 -0800 (PST) In-Reply-To: References: Date: Tue, 8 Jan 2013 15:19:00 +0530 Message-ID: Subject: Re: Embedding schema with binary encoding From: Pratyush Chandra To: user@avro.apache.org Content-Type: multipart/alternative; boundary=bcaec5524380f7a16004d2c3da84 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec5524380f7a16004d2c3da84 Content-Type: text/plain; charset=ISO-8859-1 Hi Scott, I am able to find example for json encoding with DataFileWriter which embedds schema, but unable to find DataFileWriter example for binary encoding with schema. Thanks Pratyush On Tue, Jan 8, 2013 at 2:56 PM, Scott Carey wrote: > Calling toJson() on a Schema will print it in json fom. However you most > likely do not want to invent your own file format for Avro data. > > DataFileWriter which will manage the schema for you, along with > compression, metadata, and the ability to seek to the middle of the file. > Additionally it is then readable by several other languages and tools. > > On 1/7/13 4:42 AM, "Pratyush Chandra" wrote: > > I am able to serialize with binary encoding to a file using following : > FileOutputStream outputStream = new FileOutputStream(file); > Encoder e = EncoderFactory.get().binaryEncoder(outputStream, null); > DatumWriter datumWriter = new > GenericDatumWriter(schema); > GenericRecord message1= new GenericData.Record(schema); > message1.put("to", "Alyssa"); > datumWriter.write(message1, e); > e.flush(); > outputStream.close(); > > But the output file contains only serialized data and not schema. How can > I add schema also ? > > Thanks > Pratyush Chandra > > -- Pratyush Chandra --bcaec5524380f7a16004d2c3da84 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Scott,

I am able to find example for json encoding with DataFileW= riter which embedds schema, but unable to find DataFileWriter example for b= inary encoding with schema.

Thanks
Pratyush

On Tue, Jan 8, 2013 at 2:56 PM, Scott Carey <scottcarey@apache.org> wrote:
Calling toJson() on a Schema will print it in json fom. =A0Howe= ver you most likely do not want to invent your own file format for Avro dat= a.

DataFileWriter which will manage the schema for you, al= ong with compression, metadata, and the ability to seek to the middle of th= e file. =A0 =A0Additionally it is then readable by several other languages = and tools.
I am able to serialize with binary encoding to a file using following :
=A0=A0=A0=A0=A0=A0=A0 FileOutputStream outputStream =3D new FileOutputStrea= m(file);
=A0=A0=A0=A0=A0=A0=A0 Encoder e =3D EncoderFactory.get().binaryEncoder(outp= utStream, null);
=A0=A0=A0 =A0=A0=A0 DatumWriter<GenericRecord> datumWriter =3D new Ge= nericDatumWriter<GenericRecord>(schema);=A0=A0=A0
=A0=A0=A0 =A0=A0=A0 GenericRecord message1=3D new GenericData.Record(schema= );
=A0=A0=A0 =A0=A0=A0 message1.put("to", "Alyssa");
=A0=A0=A0 =A0=A0=A0 datumWriter.write(message1, e);
=A0=A0=A0 =A0=A0=A0 e.flush();
=A0=A0=A0 =A0=A0=A0 outputStream.close();

But the output file contains only serialized data and not schema. How can I= add schema also ?

Thanks
Pratyush Chandra



--
Pratyush Chandra
--bcaec5524380f7a16004d2c3da84--