avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Teryl Taylor <teryl.tay...@gmail.com>
Subject Re: Serializing Generated objects to data file with Avro C++
Date Thu, 23 Jun 2011 18:36:44 GMT
Okay, I think I found a somewhat workable solution.   If I serialize all my
records using the avro::Writer:

Math::complex c;
    c.real = 10.0;
    c.imaginary = 20.0;

    // Writer is the object that will do the actual I/O and buffer the results
    avro::Writer writer;

    // This will invoke the writer on my object
    avro::serialize(writer, c);


I can do the following to write them out:

avro::istream is(writer->buffer());
 ofstream avroFile(outputFile.c_str(), ios::out | ios::app | ios::binary);

 if(avroFile.is_open())
    {
          avroFile << is.rdbuf();
          avroFile.close();
  }
 else
 {
        cout << "Uh oh...couldn't write out AVRO payload data..." << endl;
        exit(1);
 }

I'm having a problem reading the file back in for parsing...I do the
following:

ifstream avroFile("payload.bin", ios::in | ios::binary);

    if(avroFile.is_open())
    {
        Math::Complex blah;
        avro::ostream os;
        os << avroFile.rdbuf();
        avro::Reader p(os.getBuffer());
        while(!os.eof())
<------------------------------------------------------------------------
This doesn't work.....
        {
           avro::parse(p, blah);
           cout << "The Real Value is " << blah.real << endl;
        }

    }

How do you tell that you have reached the end of a buffer (i.e. read all the
records in a file)?

Cheers,

Teryl





On Thu, Jun 23, 2011 at 10:16 AM, Teryl Taylor <teryl.taylor@gmail.com>wrote:

> Hi everyone,
>
> I'm playing around with Avro C++ and it looks pretty neat.  I generated a
> class based on a JSON schema which was dropped in a header file.    What is
> the best way to write out instances of these classes to a file?   The
> example in the documentation shows:
>
>  Math::complex c;
>     c.real = 10.0;
>     c.imaginary = 20.0;
>
>     // Writer is the object that will do the actual I/O and buffer the results
>     avro::Writer writer;
>
>     // This will invoke the writer on my object
>     avro::serialize(writer, c);
>
>
>
> ***********************************************************************************************
>
>
> But it is unclear how to attach a file stream to the writer.   I also tried:
>
> avro::compileJsonSchema(is, validSchema);
> 		is.close();
> 		dfWriter = new avro::DataFileWriter<GenObject>(outputdir.c_str(), validSchema);
> dfWriter->write(obj);
>
>
> But this won't compile giving the error:
>
> /terylt/Projects/bitvectors/avro-cpp-1.5.1/api/Specific.hh:195: error:
> 'encode' is not a member of 'avro::codec_traits<GenObject>'
>
>
> What's the best way to write out my objects to disk?
>
>
> Regards,
>
>
> Teryl
>
>

Mime
View raw message