avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Balajee R.C" <bala...@vizexperts.com>
Subject Trouble Serializing Record from a multi-Record Avro Schema using C++
Date Fri, 17 Jun 2016 09:53:14 GMT
I am getting a runtime exception when serializing an Avro record generated
from JSON schema containing multiple records in a C++ program.

I was able to reproduce this problem with both avrocpp-1.7.7 aas well as
avrocpp-1.8.1 (latest stable release).

Here is my schema:

[
> {
>   "type" : "record",
>   "name" : "Point",
>   "doc:" : "Struct holding x, y, z values",
>   "namespace": "Geometry",
>   "fields" : [
>       {
>         "name": "x",
>         "type": "double"
>       },
>       {
>         "name": "y",
>         "type": "double"
>       },
>       {
>         "name": "z",
>         "type": "double"
>       }
>   ]
> },
> {
>   "type" : "record",
>   "name" : "Polygon",
>   "doc:" : "A collection of points and edges",
>   "namespace": "Geometry",
>   "fields" : [
>       {
>         "name": "points",
>         "type": ["Point"],
>         "doc": "Points within this polygon"
>       }
>   ]
> }
> ]


 Here is my cpp code:

#include <sstream>
> #include <fstream>
> #include <avro/Compiler.hh>
> #include "schema.h"
> int main()
> {
>     avro::ValidSchema schema;
>     std::string error;
>
>     std::ifstream inputJson("schema.json");
>     try
>     {
>         avro::compileJsonSchema(inputJson, schema);
>     }
>     catch(std::exception& err)
>     {
>         std::cout << "Error while parsing schema: " << err.what() <<
> std::endl;
>         exit(1);
>     }
>     std::cout << "Found valid JSON. Proceeding with encoding/decoding
> test" << std::endl;
>     std::stringstream output;
>     std::auto_ptr<avro::OutputStream> out =
> avro::ostreamOutputStream(output);
>     avro::EncoderPtr e = avro::jsonEncoder(schema);
>     e->init(*out);
>     Geometry::Point pt;
>     pt.x = 1.0;
>     pt.y = 2.13;
>     pt.z = 3.2;
>     avro::encode(*e, pt);
>     e->flush();
>     std::cout << "Encoded: " << std::endl  << output.str() <<
std::endl;
>     std::auto_ptr<avro::InputStream> in = avro::istreamInputStream(output);
>     avro::DecoderPtr d = avro::jsonDecoder(schema);
>     d->init(*in);
>
>     Geometry::Point c2;
>     avro::decode(*d, c2);
>     std::cout << '(' << c2.x << ", " << c2.y << ')' <<
std::endl;
>     std::cout << "Done." << std::endl;
>     return 0;
> }


 This is how I am generating C++ header from schema:

avrogencpp -i schema.json -o schema.h -n Geometry


Here is the output I get in my console on running the program:

Found valid JSON. Proceeding with encoding/decoding test
> terminate called after throwing an instance of 'avro::Exception'
>   what():  Invalid operation. Expected: Double got Union
> Aborted (core dumped)


I get the same exception if I use a "nested" schema:

[
> {
>   "type" : "record",
>   "name" : "Polygon",
>   "doc:" : "A collection of points and edges",
>   "namespace": "Geometry",
>   "fields" : [
>     {
>       "name": "points",
>       "type": ["null", {
>         "type" : "record",
>         "name" : "Point",
>         "doc:" : "Struct holding x, y, z values",
>         "namespace": "Geometry",
>         "fields" : [
>             {
>               "name": "x",
>               "type": "double"
>             },
>             {
>               "name": "y",
>               "type": "double"
>             },
>             {
>               "name": "z",
>               "type": "double"
>             }
>         ]
>       }],
>       "doc": "Points within this polygon"
>     }
>   ]
> }
> ]


And yet, the program runs fine if I use a schema with ONLY the Point record
defined, and leaving out the Polygon record:

{
>   "type" : "record",
>   "name" : "Point",
>   "doc:" : "Struct holding x, y, z values",
>   "namespace": "Geometry",
>   "fields" : [
>       {
>         "name": "x",
>         "type": "double"
>       },
>       {
>         "name": "y",
>         "type": "double"
>       },
>       {
>         "name": "z",
>         "type": "double"
>       }
>   ]
> }


With the above schema containing only Point record, I get the correct
output:

Found valid JSON. Proceeding with encoding/decoding test
> Encoded:
> {"x":1,"y":2.13,"z":3.2}
> (1, 2.13)
> Done.


Can someone point out whether the above issue I am facing is because of
something I am doing wrong? If not, I can file a bug report for the same.

Other details:
OS: Linux (Ubuntu 14.04, x86_64)
Boost version that avro was built with: 1.56
g++ version: 4.8.4

Regards,
Balajee.R.C

Mime
View raw message