hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Is DFSInputStream.read(long position,...) designed for multi threaded access?
Date Wed, 02 Mar 2011 22:10:23 GMT
On Wed, Mar 2, 2011 at 8:17 AM, Jason Rutherglen
<jason.rutherglen@gmail.com> wrote:
> Todd,
>
> Thanks for the reply.  I looked at HDFS-941 which seems to remove the
> redundant creation of BlockReaders.  That and HDFS-347 will solve some
> of the issues, however I think there's also the sendmsg() of the file
> descriptor that also needs to be implemented?

Yes, see HADOOP-6311. I have an up-to-date patch on this but have not
yet gotten to posting it since we need 347 to be redone on trunk as
well.

-Todd

> On Tue, Mar 1, 2011 at 5:58 PM, Todd Lipcon <todd@cloudera.com> wrote:
>> Hi Jason,
>>
>> Yes, this method is currently very inefficient.. HDFS-941 will
>> hopefully improve this situation, but currently there's no
>> particularly efficient way to do multithreaded random access.
>>
>> -Todd
>>
>> On Tue, Mar 1, 2011 at 4:24 PM, Jason Rutherglen
>> <jason.rutherglen@gmail.com> wrote:
>>> It's unsynchronized however it's creating a new BlockReader on each
>>> call, that seems like a problem?
>>>
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message