avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <scottca...@apache.org>
Subject Re: AVRO Path
Date Fri, 13 Jan 2012 22:04:03 GMT

On 1/13/12 11:01 AM, "Jason Rutherglen" <jason.rutherglen@gmail.com> wrote:

>It makes a lot of sense for NoSQL systems that store large AVRO blobs,
>that only require a subset of the data, where the entire blob could
>generate significant network traffic.  In a system like HBase, AVRO
>makes a lot of sense because schema evolution via HBase can be hairy,
>and the storage format is verbose.
>I would open a ticket however I do not have time allotted to work on it.

A ticket with an explanation of the use cases and some high level ideas is
fine.  It centralizes the discussion on the idea, and if someone stumbles
upon it later who has time to work on it then its nice to have an easy
reference location for those ideas.

>On Thu, Jan 12, 2012 at 10:05 AM, Scott Carey <scottcarey@apache.org>
>> There are no plans that I know of currently, although the topic came up
>> two times in separate conversations last night at the SF Hadoop MeetUp.
>> I think an ability to extract a subset of a schema from a larger one and
>> read/write/transform data accordingly makes a lot of sense. Currently,
>> Avro spec allows for schema resolution which is sort of a degenerate
>> schema extraction/transformation at the record level without the ability
>> to address or extract nested elements.  An addition to the spec for
>> describing other schema extractions may be useful.  Further discussion
>> should probably be in a JIRA ticket or at least on the dev list.
>> -Scott
>> On 1/10/12 1:02 PM, "Jason Rutherglen" <jason.rutherglen@gmail.com>
>>>Are there plans for (or is there already) an AVRO Path implementation
>>>(like XPath, or JSON Path).

View raw message