hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
Date Wed, 02 Mar 2011 22:42:02 GMT
On Wed, Mar 2, 2011 at 2:38 PM, Andrew Purtell <apurtell@apache.org> wrote:
> I looked at HADOOP-6311 recently when deciding if I wanted to port the full FD-passing
bits of HDFS-347 onto CDH3B3. I see Owen changed his -1 to -0, but that's not really comforting.
How likely is this to go in? Would this go in to CDH if I posted a patch for HADOOP-6311 and

Regarding trunk, that's for the general community to decide. I think
the increased interest in HBase across the community might have
changed people's minds - lots of people are now agreeing that 347 is a
great help for HBase. The fd-passing approach is so far the only one
I've seen proposed that addresses security.

Regarding CDH, this probably isn't the best place for discussing that :)

> My aim here is a fast local read path that respects security, obviously, something that
I won't be supporting myself in production against an increasingly diverging upstream.

Agreed completely.


> --- On Wed, 3/2/11, Todd Lipcon <todd@cloudera.com> wrote:
>> From: Todd Lipcon <todd@cloudera.com>
>> Subject: Re: Is DFSInputStream.read(long position,...) designed for multi threaded
>> To: "Jason Rutherglen" <jason.rutherglen@gmail.com>
>> Cc: hdfs-user@hadoop.apache.org
>> Date: Wednesday, March 2, 2011, 2:10 PM
>> On Wed, Mar 2, 2011 at 8:17 AM, Jason
>> Rutherglen
>> <jason.rutherglen@gmail.com>
>> wrote:
>> > Todd,
>> >
>> > Thanks for the reply.  I looked at HDFS-941 which seems to remove the
>> > redundant creation of BlockReaders.  That and HDFS-347 will solve some
>> > of the issues, however I think there's also the sendmsg() of the file
>> > descriptor that also needs to be implemented?
>> Yes, see HADOOP-6311. I have an up-to-date patch on this but have not
>> yet gotten to posting it since we need 347 to be redone on trunk as
>> well.
>> -Todd

Todd Lipcon
Software Engineer, Cloudera

View raw message