avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raihan Jamal <jamalrai...@gmail.com>
Subject Re: Apache Avro Encoding types?
Date Thu, 19 Sep 2013 03:39:51 GMT
Thanks Connor. That makes sense... That whats we are planning to do. We
will have Cassandra column which will have composite columns. And those
composite column will have-

Avro-Binary-Encoded_value   Avro-Schema-Name-For-This particular-column

So when we will retrieve the data for a particular USER-ID for the above
column, we will know its Schema-Name, then we can pull the actual schema
basis on this schema-name, and then we can deserialize the
Avro-Binary-Encoded_value using that schema.

*Raihan Jamal*

On Wed, Sep 18, 2013 at 7:15 PM, Connor Doyle <connor.p.d@gmail.com> wrote:

> Hi Raihan,
> One strategy may be to store a cache of schemata elsewhere and then write
> just enough information (version number, md5, or similar) to look up the
> right schema version adjacent to your avro/binary blob column.  This would
> impose little overhead and still allow you to take advantage of Avro's
> built-in schema resolution to read old serialized data.
> --
> Connor
> On Sep 18, 2013, at 20:03, raihan26 <jamalraihan@gmail.com> wrote:
> Thanks a lot Doug. So that means, If I need to store avro binary encoded
> data in Cassandra column family, then the actual schema won't get stored
> with that? Right? I need to have the schema through some external means
> whenever I am reading that binary encoded data. Right?
> *Raihan Jamal*
> On Wed, Sep 18, 2013 at 2:28 PM, Doug Cutting [via Apache Avro] <[hidden
> email] <http://user/SendEmail.jtp?type=node&node=4028216&i=0>> wrote:
>> On Wed, Sep 18, 2013 at 1:53 PM, Raihan Jamal <[hidden email]<http://user/SendEmail.jtp?type=node&node=4028215&i=0>>
>> wrote:
>> > Suppose if I am going with Binary encoding route instead of JSON then
>> does
>> > the schema also gets stored in that binary encoding format always?
>> The schema is stored in Avro data files.  Avro RPC also manages
>> schemas for you.  But if you store Avro binary-encoded data in some
>> other container then you may need to keep track of the schema
>> yourself.
>> Doug
>> ------------------------------
>>  If you reply to this email, your message will be added to the
>> discussion below:
>> http://apache-avro.679487.n3.nabble.com/Apache-Avro-Encoding-types-tp4028194p4028215.html
>>  To start a new topic under Avro - Users, email [hidden email]<http://user/SendEmail.jtp?type=node&node=4028216&i=1>
>> To unsubscribe from Apache Avro, click here.
>> NAML<http://apache-avro.679487.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> ------------------------------
> View this message in context: Re: Apache Avro Encoding types?<http://apache-avro.679487.n3.nabble.com/Apache-Avro-Encoding-types-tp4028194p4028216.html>
> Sent from the Avro - Users mailing list archive<http://apache-avro.679487.n3.nabble.com/Avro-Users-f679479.html>at
> Nabble.com.

View raw message