avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Balashov <ibalas...@gmail.com>
Subject Re: Extracting records as bytes from avro file
Date Thu, 16 Oct 2014 16:58:29 GMT
Thanks for the clarification, Doug.

Could adding length prefix be considered feasible enough to file a
feature request?


2014-10-15 21:15 GMT+04:00 Doug Cutting <cutting@apache.org>:
> There is no method to read individual records as binary since they're
> not delimited nor length-prefixed.  Instead you can get a block of
> records as binary and the count of records in the block and pass that
> to a deserializer that parses individual records.
>
> Doug
>
> On Wed, Oct 15, 2014 at 4:47 AM, Ivan Balashov <ibalashov@gmail.com> wrote:
>> Hi,
>>
>> Imagine a scenario where one reads avro file and distributes
>> individual records (aka datums). Suppose the schema is known, and we
>> just need to pass records as bytes to another system for further
>> processing.
>>
>> Given the above, is there a way to read individual records as bytes
>> without deserialising them first?
>>
>> As far as the documentation goes it can be easily done with blocks,
>> but it doesn't mention records.
>>
>> Just to be more clear, there is a method which allows to write encoded
>> datum to stream
>> org.apache.avro.file.DataFileWriter#appendEncoded
>>
>> Is there a similar method for reading records as bytes?
>>
>> Thanks,

Mime
View raw message