avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Getting started with Avro + Reading from an Avro formatted file
Date Tue, 24 Jan 2012 21:06:47 GMT

(Forgot to reply to this before)

On Wed, Jan 25, 2012 at 1:07 AM, selvi k <gridsngators@gmail.com> wrote:
> 3. With regards to the two suggested ways, would either of these techniques
> allow me to filter my data records using some sort of a condition on a
> field?(or a few fields)  If not it seems like I would have to resort to
> first grepping the log file with the condition I want, and then using either
> of these two techniques to convert to CSV file. This would still be much
> better than what I am doing now, which is through not-so-pretty awk
> invocations to retrieve the fields I need (after the initial grep). But if
> the existing API, allows me to scan through the log file and specify
> conditions for fields, it might be much more efficient. I can imagine that I
> might have to use the low-level API and write a program to do this, but I am
> not sure at this point how to get started on this.

$ avro --help has some options that can help you out.

For "avro cat", the following may help:

--fields=FIELDS     fields to show, comma separated (show all by default)

But no, the utility does not provide a way to filter anything out. Its
a mere reader with some extensibility on fields/format. You'd have to
do filtering via your own full-fledged reader program, or via Bash
using "avro cat" and grep/etc.

Harsh J
Customer Ops. Engineer, Cloudera

View raw message