hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: Does hadoop 0.18.3 support thrift ?
Date Wed, 30 Sep 2009 03:30:29 GMT
On Tue, Sep 29, 2009 at 11:20 PM, Todd Lipcon <todd@cloudera.com> wrote:
> Ah, right, it was introduced in 0.19.
>
> It shouldn't be too tough to compile against 0.18 - the API changes were
> minimal and it's an external client so internal changes shouldn't have any
> effect.
>
> I'd recommend pulling the source from 0.20, moving it into the contrib
> directory, and hacking away.
>
> If you aren't comfortable in Java and with build.xml, though, you're
> probably out of luck - it's unlikely to work without a bit of coercion.
>
> -Todd
>
>
> On Tue, Sep 29, 2009 at 8:11 PM, Jeff Zhang <zjffdu@gmail.com> wrote:
>
>> But I do not found the thrift in hadoop 0.18.3.
>> Can I use other version of hadoop's thrift for my cluster which is built
>> upon hadoop 0.18.3
>>
>>
>> On Wed, Sep 30, 2009 at 1:56 AM, Todd Lipcon <todd@cloudera.com> wrote:
>>
>> > Hi Jeff,
>> >
>> > The contrib/thriftfs module is probably what you're looking for.
>> >
>> > I don't know of a lot of people using it, but it does provide a
>> proxy-style
>> > thrift access to HDFS.
>> >
>> > Be aware that performance will not be great since it introduces several
>> > more
>> > copies in the pipeline, and Thrift was never designed for transferring
>> > large
>> > amounts of data.
>> >
>> > -Todd
>> >
>> > On Mon, Sep 28, 2009 at 10:37 PM, Jeff Zhang <zjffdu@gmail.com> wrote:
>> >
>> > > Hi all,
>> > >
>> > >
>> > > My hadoop cluster is using hadoop 0.18.3. And I'd  like to using python
>> > to
>> > > access hdfs. But I did not found thrift in hadoop 0.18.3.
>> > >
>> > > Anybody know does hadoop 0.18.3 support thrift? If not, is there any
>> > other
>> > > method to access hdfs using python ?
>> > >
>> > >
>> > > Thank you in advance.
>> > >
>> > > Jeff zhang
>> > >
>> >
>>
>

I did this at one point. The code reference append and some other
things not in 18.0 but It does work.

Mime
View raw message