avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bhuvana Bellala <Bhuvana.Bell...@cormac-corp.com>
Subject RE: COnverting CSV files to avro and back to text
Date Wed, 09 Jul 2014 15:13:19 GMT
Thanks for all the help! I will try both ways and let you all know what worked best.

From: Sean Busbey [mailto:busbey@cloudera.com]
Sent: Wednesday, July 09, 2014 3:41 AM
To: user@avro apache. org
Subject: Re: COnverting CSV files to avro and back to text

You could consider using the KiteSDK[1]. It uses Avro schemas to describe datasets, can infer
an avro schema from CSV headers, and includes tools for importing from CSV to Avro for storage.
There's a tool demo that can walk you through getting things into avro and displaying a plain
text version[2].

The current MapReduce support is a first pass, but is enough to start playing on[3]. Unfortunately,
I don't think it has a demo walkthrough yet. The next release is supposed to be more user
friendly.


[1]: http://kitesdk.org/docs/current/kite-data/guide.html
[2]: http://kitesdk.org/docs/current/usingkiteclicreatedataset.html
[3]: http://kitesdk.org/docs/current/apidocs/org/kitesdk/data/mapreduce/DatasetKeyInputFormat.html

On Mon, Jul 7, 2014 at 4:13 PM, Bhuvana Bellala <Bhuvana.Bellala@cormac-corp.com<mailto:Bhuvana.Bellala@cormac-corp.com>>
wrote:
Hi,

  I want convert a CSV file into avro format so that I can compress data and do some map reduce
things in Hadoop. And then I want to convert the avro data back to text. Are there any tools
to do this? I know I could use hive to do this but I want to know if there are any other ways
to do this? Thank!

[Cormac_logo]

Bhuvana Bellala| Intern
________________________________
13921 Park Center Road, Suite 180, Herndon VA 20171|
CMMI Maturity Level 3 Rated| GSA Schedule 70| SBA SDB, 8(a)|
www.cormac-corp.com<http://www.cormac-corp.com/>

This email and any files transmitted with it are confidential and intended solely for the
use of the individual or entity to whom they are addressed.  If you are not the named addressee
you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately
by e-mail if you have received this e-mail by mistake and delete this e-mail from your system.
If you are not the intended recipient you are notified that disclosing, copying, distributing
or taking any action in reliance on the contents of this information is strictly prohibited.





--
Sean
Mime
View raw message