Return-Path: X-Original-To: apmail-avro-user-archive@www.apache.org Delivered-To: apmail-avro-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CBC36178C4 for ; Wed, 20 May 2015 13:10:55 +0000 (UTC) Received: (qmail 13405 invoked by uid 500); 20 May 2015 13:10:55 -0000 Delivered-To: apmail-avro-user-archive@avro.apache.org Received: (qmail 13338 invoked by uid 500); 20 May 2015 13:10:55 -0000 Mailing-List: contact user-help@avro.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@avro.apache.org Delivered-To: mailing list user@avro.apache.org Received: (qmail 13328 invoked by uid 99); 20 May 2015 13:10:55 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 May 2015 13:10:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 8DA421A3799 for ; Wed, 20 May 2015 13:10:54 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.991 X-Spam-Level: ** X-Spam-Status: No, score=2.991 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=3, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id xqCuojSIx6oe for ; Wed, 20 May 2015 13:10:42 +0000 (UTC) Received: from DUB004-OMC3S8.hotmail.com (dub004-omc3s8.hotmail.com [157.55.2.17]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id D9E9840E1C for ; Wed, 20 May 2015 13:10:41 +0000 (UTC) Received: from DUB130-W10 ([157.55.2.8]) by DUB004-OMC3S8.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.22751); Wed, 20 May 2015 06:10:13 -0700 X-TMN: [h7sQUYzF7MH+beSAHzN+71Kl69sCvY2X] X-Originating-Email: [thepedrau@hotmail.com] Message-ID: Content-Type: multipart/alternative; boundary="_97e933a7-2bc1-4d72-bcb1-00300ce78c84_" From: =?iso-8859-1?B?UGllcnJlIGRlIEZy6W0=?= To: "user@avro.apache.org" Subject: RE: Not able to load avro schema fully with all its contents Date: Wed, 20 May 2015 13:10:13 +0000 Importance: Normal In-Reply-To: References: ,,,,<183093874.2064282.1432060676169.JavaMail.yahoo@mail.yahoo.com>, MIME-Version: 1.0 X-OriginalArrivalTime: 20 May 2015 13:10:13.0805 (UTC) FILETIME=[4EF93DD0:01D092FE] --_97e933a7-2bc1-4d72-bcb1-00300ce78c84_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello=2C I posted the patch for the trunk branch of the git there (for it to be revi= ewed):https://issues.apache.org/jira/browse/AVRO-1256 Pierre From: thepedrau@hotmail.com To: user@avro.apache.org Subject: RE: Not able to load avro schema fully with all its contents Date: Wed=2C 20 May 2015 10:08:22 +0000 =0A= =0A= =0A= Hello=2C Sam is right in his previous answer.More precisely=2C the field "doc" is re= ad by the Compiler=2C but not stored at the moment in the Node object. The = reason might be that the field "doc" is optional is the avro specification = (see: https://avro.apache.org/docs/1.7.7/spec.html=2C Complex types). If you want to store the field doc=2C you'll have to modify the source code= yourself to:- create a new member "doc" in the Node API (Node.hh)=2C- stor= e the doc field in Node as it is read by the Compiler (Compiler.cc)=2C- ser= ialize the field doc in NodeImpl.cc I did a patch for my own use were I store and read fields "doc" for a NodeR= ecord=2C and I serialize fields doc for the root Node of a NodeRecord. You can find it at:the corresponding branch (created for the patch):https:/= /github.com/pidefrem/avro/tree/branch-1.7-specificrecord the corresponding commit for the field doc:https://github.com/pidefrem/avro= /commit/795a0805b8ea8d3228bd92a483c9cbb405e11a62 Rem: if you want to serialize all fields doc of a NodeRecord=2C just change= line 195 of NodeImpl.cc from if (depth =3D=3D 1 && getDoc().size()) { to if (getDoc().size()) { (Maybe my patch could be added in the trunk of the source code if it is use= ful?) Hope this helps. Pierre Date: Tue=2C 19 May 2015 18:37:56 +0000 From: sgroth@yahoo-inc.com To: user@avro.apache.org Subject: Re: Not able to load avro schema fully with all its contents Just a guess=2C but I would assume that the schema object only stores field= s that it cares about. This would exclude your docs. If you want to know fo= r sure=2C the source code is here: https://github.com/apache/avro/tree/trun= k/lang/c%2B%2B =20 Sam On Tuesday=2C May 19=2C 2015 1:13 PM=2C Check Peck wrote: =20 Can anyone help me with this?On Mon=2C May 18=2C 2015 at 2:04 PM=2C Check = Peck wrote:Does anyone have any idea on this why = it is behaving like this?On Mon=2C May 18=2C 2015 at 1:03 PM=2C Check Peck = wrote:And this is my to_string method I forgot to= provide.std::string DataSchema::to_string() const{ ostringstream os=3B = if (valid()) { os << "JSON data: "=3B m_schema.toJson(o= s)=3B } return os.str()=3B}On Mon=2C May 18=2C 2015 at 12:54 PM=2C = Check Peck wrote:I am working with Apache Avro in= C++ and I am trying to load avro schema by using Avro C++ library. Everyth= ing works fine without any issues=2C only problem is - I have few "doc" in = my Avro schema which is not getting shown at all in my AvroSchema when I tr= y to load it and also print it out. DataSchema_ptr schema_data(new DataS= chema())=3B schema_data->m_schema =3D load(avro_schema_file_name.c_str()= )=3B const avro::NodePtr node_data_ptr =3D schema_data->m_schema.root()= =3B if (node_data_ptr && node_data_ptr->hasName()) { // is the= re any problem with this node_data_ptr usage here? schema_data->m_na= me =3D node_data_ptr->name().fullname().c_str()=3B // this li= ne prints out whole AVRO but it doesn't have doc which is there in my AVRO = cout<<"File String : " << schema_data->to_string() << endl=3B } = Here "m_schema" is "avro::ValidSchema m_schema=3B" Can anyone help me wi= th this. In general I don't see my doc which I have in Avro Schema getting = shown when I print it out.=0A= =0A= =0A= = --_97e933a7-2bc1-4d72-bcb1-00300ce78c84_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hello=2C

I po= sted the patch for the trunk branch of the git there (for it to be reviewed= ):

Pierre

From: thepedra= u@hotmail.com
To: user@avro.apache.org
Subject: RE: Not able to load = avro schema fully with all its contents
Date: Wed=2C 20 May 2015 10:08:2= 2 +0000

=0A= =0A= =0A=
Hello=2C

Sam is right in his previous a= nswer.
More precisely=2C =3Bthe field "doc" is read by the Compiler=2C but not stored at the moment in= the Node object. The reason might be that the field "doc" is optional is t= he avro specification (see: =3Bhttps= ://avro.apache.org/docs/1.7.7/spec.html=2C Complex types).

If you want to store = the field doc=2C you'll have to modify the source code yourself to:
- create a new member "doc" in the Node API (Node.hh)=2C
- sto= re the doc field in Node as it is read by the Compiler (Compiler.cc)=2C
- serialize the field doc in NodeImpl.cc

I = did a patch for my own use were I store and read fields "doc" for a NodeRec= ord=2C and I serialize fields doc for the root Node of a NodeRecord.
<= div>
You can find it at:
the corresponding branch (= created for the patch):

the = corresponding commit for the field doc:

Rem: if you want to serializ= e all fields doc of a NodeRecord=2C just change line 195 of NodeImpl.cc fro= m
if (depth =3D=3D 1 &=3B&=3B getDoc().size if= (getDoc().= size()) {

(Maybe my pat= ch could be added in the trunk of the source code if it is useful?)

Hope this helps.

Pierre

<= div>
Date: Tue=2C 19 May 2015 18:37:56 +0000
F= rom: sgroth@yahoo-inc.com
To: user@avro.apache.org
Subject: Re: Not a= ble to load avro schema fully with all its contents

Just a guess=2C but I would assume that the schema object only stores fiel= ds that it cares about. This would exclude your docs. If you want to know f= or sure=2C the source code is here: =3Bhttps://github.com/apache/avro/tree/trunk/lang= /c%2B%2B


Sam<= /div>


On Tuesday=2C May 19=2C 2015 1:13 PM=2C Check Peck <=3Bc= omptechgeeky@gmail.com>=3B wrote:


= Can anyone help me with this?

On Mon=2C May 18=2C 2015 at 2:04 PM=2C Check Peck <=3Bcomptechgeeky@gmail.com>=3B wrote:=
Does= anyone have any idea on this why it is behaving like this?

On Mon=2C May 18=2C 2015 at 1:03 PM=2C = Check Peck <=3Bcomptechgeeky@gmail.co= m>=3B wrote:
And this is my to_string method I forgot to provide.<= br clear=3D"none">
std::string DataSchema::to_string() co= nst
{
 =3B =3B =3B ostrings= tream os=3B
 =3B =3B =3B if (valid())
 =3B =3B =3B {
 =3B =3B&nb= sp=3B =3B =3B =3B =3B os <=3B<=3B "JSON data: "=3B
 =3B =3B =3B =3B =3B =3B =3B m_sc= hema.toJson(os)=3B =3B
 =3B =3B =3B } =3B =3B =3B return os.str()=3B

}


On Mon=2C May 18=2C 2015 at 12:54 PM=2C Check Peck <=3Bcomptechgeeky@gmail.com>=3B wrote:
I am wor= king with Apache Avro in C++ and I am trying to load avro schema by using A= vro C++ library. Everything works fine without any issues=2C only problem i= s - I have few "doc" in my Avro schema which is not getting shown at all in= my AvroSchema when I try to load it and also print it out.

 =3B =3B =3B DataSchema_ptr schema_data(n= ew DataSchema())=3B
 =3B =3B =3B schema_data-= >=3Bm_schema =3D load(avro_schema_file_name.c_str())=3B
 =3B =3B =3B const avro::NodePtr node_data_ptr =3D schema_data= ->=3Bm_schema.root()=3B
 =3B =3B =3B if (no= de_data_ptr &=3B&=3B node_data_ptr->=3BhasName())
 =3B =3B =3B {
 =3B =3B =3B = =3B =3B =3B =3B // is there any problem with this node_data_ptr= usage here?
 =3B =3B =3B =3B =3B&nbs= p=3B =3B schema_data->=3Bm_name =3D node_data_ptr->=3Bname().fullna= me().c_str()=3B
 =3B =3B =3B =3B =3B =  =3B
 =3B =3B =3B =3B =3B =3B=  =3B // this line prints out whole AVRO but it doesn't have doc which i= s there in my AVRO
 =3B =3B =3B =3B = =3B =3B =3B cout<=3B<=3B"File String : " <=3B<=3B schema_da= ta->=3Bto_string() <=3B<=3B endl=3B
 =3B = =3B =3B }
 =3B  =3B
Here "m= _schema" is "avro::ValidSchema m_schema=3B"
 =3B &nbs= p=3B
Can anyone help me with this. In general I don't see= my doc which I have in Avro Schema getting shown when I print it out.=0A=

=0A=

=0A=

<= br>
= --_97e933a7-2bc1-4d72-bcb1-00300ce78c84_--