avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Wasserman <ewasser...@247-inc.com>
Subject RE: Is there a way to conditionally read Avro data?
Date Fri, 16 Aug 2013 22:47:05 GMT
If you define you records like this (this is in the Avro IDL lang. for brevity)

If you write your records with a schema like this:

record R {

    Header header;

    Body body;


Then you can read with a schema like this:

  record RSansBody {

    Header header;


And the Avro libraries will read the header part (in which your "type" would reside) and effectively
skip the body part.

From: Anna Lahoud <annalahoud@gmail.com>
Sent: Friday, August 16, 2013 12:23 PM
To: user@avro.apache.org
Subject: Is there a way to conditionally read Avro data?

I am wondering if there is a way that I can avoid reading all of an item in an Avro file,
based on some of the data that I have already read. For instance, say I have a datum where
I know that if it's 'type' value is a 'ComputerVirus', and that I do not want to touch the
remaining fields. Is there a way to 'move on' and get the next datum, without touching the
remainder of the scary datum? I would call it a 'conditional read' in that I only want to
fully read the datum if the datum meets some criteria.


View raw message