avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From felix gao <gre1...@gmail.com>
Subject Re: How to get started with examples on avro
Date Fri, 28 Jan 2011 21:53:38 GMT
The goal to convert to avro is for us to use the splittable property so we
can have some type of  compressed data for huge log files so we can save
some hdfs disk spaces.

On Fri, Jan 28, 2011 at 1:02 PM, Anand Padmanaban <apadmanaban@splunk.com>wrote:

> Meta question.  I see avro is the means, what is the end goal?  What do you
> want to do with the data after converting it to avro?
>
> > -----Original Message-----
> > From: felix gao [mailto:gre1600@gmail.com]
> > Sent: Friday, January 28, 2011 12:30 PM
> > To: user@avro.apache.org
> > Subject: Re: How to get started with examples on avro
> >
> > Thanks for the quick reply.  I am interested in doing this through the
> java implementation and I would like to do it in parallel that
> > utilizes the mapreduce framework.
> >
> >
> > On Fri, Jan 28, 2011 at 12:22 PM, Harsh J <qwertymaniac@gmail.com>
> wrote:
> >
> >
> >       Based on the language you're targeting, have a look at its
> test-cases
> >       available on the in the project's version control:
> >       http://svn.apache.org/repos/asf/avro/trunk/lang/ [You can check it
> out
> >       via SVN, or via Git mirrors]
> >
> >       Another good resource on the ends of Avro (Data and RPC) is by
> phunt
> >       at http://github.com/phunt/avro-rpc-quickstart#readme
> >
> >       I had written a python data-file centric snippet for Avro a while
> ago
> >       at my blog; it may help if you're looking to get started with
> Python
> >       (although it does not cover all aspects, which the functions in the
> >       available test cases for lang/python do):
> >
> http://www.harshj.com/2010/04/25/writing-and-reading-avro-data-files-using-python/
> >
> >
> >       On Sat, Jan 29, 2011 at 1:34 AM, felix gao <gre1600@gmail.com>
> wrote:
> >       > Hi all,
> >       > I am trying to convert a lot of our existing logs into avro
> format in
> >       > hadoop.  I am not sure if there are any examples to follow.
> >       > Thanks,
> >       > Felix
> >
> >
> >
> >
> >       --
> >       Harsh J
> >       www.harshj.com
> >
> >
>
>

Mime
View raw message