avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Read specific record from DataFileReader for avro files
Date Tue, 13 Aug 2013 17:38:30 GMT
Avro data files don't easily permit random access to records.  One
approach is that you could index the result of the previous
DataFileWriter#sync() of each record, then use DataFileReader#seek()
and scan for the desired record.

Also, please consider using SortedKeyValueFile.

http://avro.apache.org/docs/current/api/java/org/apache/avro/hadoop/file/SortedKeyValueFile.html

Doug

On Tue, Jun 4, 2013 at 9:32 AM, amit nanda <amitwip@gmail.com> wrote:
> Hi,
>
> I have avro files that have a number of records in each file, we generate
> indexes using Lucene for these files, and now depending upon the index
> matches I want to read some specific records from the avro file.
>
> I am using DataFileReader for this, but i am not able to find a way to read
> random records. ANy help is much appreciated.
>
>
>
>

Mime
View raw message