hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: datanode write forwarding
Date Tue, 18 Dec 2012 22:31:21 GMT
Can't speak for their explanation (not sure what doc you seem to
be referring to?) but the behavior of mirror-first-disk-next has been
there since 0.20 at least - so yes, 1.0 too has the same behavior.

If you speak of the doc
http://hadoop.apache.org/docs/stable/hdfs_design.html#Replication+Pipelining,
that seems to be a bit outdated overall. We could change the text -
mind filing a JIRA and doing so?

On Wed, Dec 19, 2012 at 3:49 AM, Chris Embree <cembree@gmail.com> wrote:
> Harsh,
>
> Is that a change from 1.0 code?  Hortonworks explains it a little
> differently.
>
> Thanks for the details and pointer to the code.
>
> Chris (another one)
>
> On Dec 18, 2012 5:14 PM, "Harsh J" <harsh@cloudera.com> wrote:
>>
>> Hi,
>>
>> The received write packet is directly socket-written to the next
>> node's receiver (async process - we don't wait for it to hit the disk
>> there), and then written onto the disk of self().
>>
>> You may see HDFS classes BlockReceiver and PacketReceiver in trunk for
>> more information.
>>
>> On Wed, Dec 19, 2012 at 2:26 AM, Jay Vyas <jayunit100@gmail.com> wrote:
>> > Hi :
>> >
>> > Do data nodes, when they receive write requests, forward the request to
>> > the
>> > next datanode in the chain after appending to a file? Or rather, do the
>> > write and forward asynchronously and at the same time?
>> >
>> > --
>> > Jay Vyas
>> > http://jayunit100.blogspot.com
>>
>>
>>
>> --
>> Harsh J



--
Harsh J

Mime
View raw message