hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Hammerbacher" <jeff.hammerbac...@gmail.com>
Subject Re: Reading and writing Thrift data from MapReduce
Date Wed, 03 Sep 2008 04:37:28 GMT
Hey Juho,

You should check out Hive
(https://issues.apache.org/jira/browse/HADOOP-3601), which was just
committed to the Hadoop trunk today. It's what we use at Facebook to
query our collection of Thrift-serialized logfiles. Inside of the Hive
code, you'll find a pure-Java (using JavaCC) parser for
Thrift-serialized data structures.

Regards,
Jeff

On Tue, Sep 2, 2008 at 6:57 AM, Stuart Sierra <mail@stuartsierra.com> wrote:
> On Tue, Sep 2, 2008 at 3:53 AM, Juho Mäkinen <juho.makinen@gmail.com> wrote:
>> What's the current status of Thrift with Hadoop? Is there any
>> documentation online or even some code in the SVN which I could look
>> into?
>
> I think you have two choices: 1) wrap your Thrift code in a class that
> implements Writable, or 2) use Thrift to serialize your data to byte
> arrays and store them as BytesWritable.
> -Stuart
>

Mime
View raw message