avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: COnverting CSV files to avro and back to text
Date Wed, 09 Jul 2014 07:40:55 GMT
You could consider using the KiteSDK[1]. It uses Avro schemas to describe
datasets, can infer an avro schema from CSV headers, and includes tools for
importing from CSV to Avro for storage. There's a tool demo that can walk
you through getting things into avro and displaying a plain text version[2].

The current MapReduce support is a first pass, but is enough to start
playing on[3]. Unfortunately, I don't think it has a demo walkthrough yet.
The next release is supposed to be more user friendly.


[1]: http://kitesdk.org/docs/current/kite-data/guide.html
[2]: http://kitesdk.org/docs/current/usingkiteclicreatedataset.html
[3]:
http://kitesdk.org/docs/current/apidocs/org/kitesdk/data/mapreduce/DatasetKeyInputFormat.html


On Mon, Jul 7, 2014 at 4:13 PM, Bhuvana Bellala <
Bhuvana.Bellala@cormac-corp.com> wrote:

>  Hi,
>
>
>
>   I want convert a CSV file into avro format so that I can compress data
> and do some map reduce things in Hadoop. And then I want to convert the
> avro data back to text. Are there any tools to do this? I know I could use
> hive to do this but I want to know if there are any other ways to do this?
> Thank!
>
>
>
> [image: Cormac_logo]
>
>
>
> *Bhuvana Bellala| Intern*
>  * ------------------------------ *
>
> *13921 Park Center Road, Suite 180, Herndon VA 20171|*
>
> *CMMI Maturity Level 3 Rated| GSA Schedule 70| SBA SDB, 8(a)*|
>
> www.cormac-corp.com
>
>
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you are not the named addressee you should not disseminate, distribute
> or copy this e-mail. Please notify the sender immediately by e-mail if you
> have received this e-mail by mistake and delete this e-mail from your
> system. If you are not the intended recipient you are notified that
> disclosing, copying, distributing or taking any action in reliance on the
> contents of this information is strictly prohibited.
>
>
>
>
>



-- 
Sean

Mime
View raw message