hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: Best way to write files to hdfs (from a Python app)
Date Thu, 12 Aug 2010 13:43:49 GMT
On 08/12/2010 08:01 AM, Bjoern Schiessle wrote:
> Hey Jeff,
>
> On Wed, 11 Aug 2010 10:40:29 -0700 Jeff Hammerbacher wrote:
>> You also mention that your app will be accessing data stored in HBase.
>> There's a Python client for the Avro HBase gateway at
>> http://github.com/hammer/pyhbase. If you try it out, let me know how it
>> goes.
>
> What's the difference between Avro and Thrift? Are there any specific
> reasons to prefer one of the other?
>
> I tried to find some documentation about Avro, but it seems that this is
> a quite new project.
>
> best wishes,
> Björn

This blog post is a good intro:

http://www.searchenginecaffe.com/2009/07/hadoop-data-serialization-battle.html

Avro is going to be supported natively in Hadoop going forward, so if 
you're on the fence, I'd choose Avro.

I've been using Avro for about a month now (just for serialization, not 
RPC) and I've been pretty happy with it.

HTH,

DR

Mime
View raw message