hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wang shu <uchicagoshuw...@gmail.com>
Subject Re: Question regarding streamer thread implementation at DFSOutputStream.java
Date Tue, 03 Jan 2017 18:07:33 GMT
Hi Manjunath,

I see. Thanks a lot.

Best,

Shu

2017-01-03 11:47 GMT-06:00 Manjunath A <manjunath.tech.a@gmail.com>:

> Hi Shu,
>
> Apologies for dealy in response.
>
> Close packet is something which is the last packet of the current block
> after writing which , the current block is closed and a new pipeline is
> setup for the next block.
>
> Your understanding of the one.isLastPacketInBlock() is correct.
>
> Regards,
> Manjunath
>
> On Wed, Dec 14, 2016 at 9:37 PM, wang shu <uchicagoshuwang@gmail.com>
> wrote:
>
>> Hi all,
>>
>> I have a question regarding the streamer thread (Line 382-575)
>>  implementation at DFSOutputStream.java(http://gr
>> epcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/had
>> oop-hdfs/2.7.1/org/apache/hadoop/hdfs/DFSOutputStream.java#D
>> FSOutputStream.DataStreamer.run%28%29).
>>
>>
>> What's the logic of line 468-485 and line 535-548, why it check the
>> packet is the last one twice and the comments are different, one is "
>> // wait for all data packets have been successfully acked" and another
>> is "// wait for the close packet has been acked".  What's the close
>> packet?
>>
>> My understanding of this is:
>> If a packet is not the last in block, then the packet is sent directly,
>> and no need to wait for ACK.
>> If a packet is the last one, then it will first wait for all previous
>> packet to be ACKed, and then it will send the last packet in this block(
>> called close packet, I guess?), and then will need wait the last packet to
>> be ACKed.
>>
>> Thanks a lot for your help.
>>
>>
>> Best,
>>
>> Shu
>>
>
>

Mime
View raw message