avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arunasalam G <zealousa...@gmail.com>
Subject Re: Doubt in a AVRO scenario
Date Wed, 11 Feb 2015 07:24:06 GMT
Hi Arvind,

Thank you very much for the timely response.

Our scenario is we have stored the data with schema added to it.

I would like to make it more simple without bringing the Hbase into

We have an Avro data object which has both data and schema and is
serialized to Byte Array.
Is there any way to retrieve the schema from this ByteArray object?

Lets assume that we don't know what schema is present in the incoming
I could find that for an AVRO data file, its possible to retrieve the
schema from the file and similarly, is there any way for retrieving the
schema from a serialized byte array object?

Thanks in advance.

Arun G

On Wed, Feb 11, 2015 at 12:35 PM, Arvind Kalyan <base16@gmail.com> wrote:

> Schema is not stored along with data. You either need to store it as part
> of the data (hbase value: schema_id + avrodata; and map schema_id to a
> schema somehow in your own code) if you have different schema for different
> rows/cells, or you can keep the schema separately for the whole hbase db if
> all rows are expected to have the same schema.
> On Tue, Feb 10, 2015 at 10:48 PM, Arunasalam G <zealousarun@gmail.com>
> wrote:
>> Hi,
>> I am new to AVRO and have a doubt in a scenario. Kindly requesting you to
>> help me on this.
>>     1. An AVRO object is constructed and is serialized and stored as
>> value in HBASE table.
>>     2. While retrieving it, we are retrieving the stored Byte Array
>> value.
>> Is it possible to retrieve the schema directly from the Byte Array value
>> just like we retrieve schema from a file?
>> Because we don't want to initialize the DatumReader object with schema
>> initialized.
>> I find that AVRO supports retrieving a schema from an AVRO file using
>> DataFileReader wherein, the schema can be obtained from the avro file and
>> hence, DatumReader can be initialized without schema.
>> But in our case, we needed to initialize the Reader without schema and
>> hence, is there any way to retrieve the schema stored in a serialized byte
>> array object?
>> I would be really greatful if you take a look at the following scenario.
>> Thanks in advance.
>> Regards,
>> Arun G
> --
> Arvind Kalyan
> http://www.linkedin.com/in/base16
> cell: (408) 761-2030

View raw message