hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manjunath A <manjunath.tec...@gmail.com>
Subject Re: Question regarding streamer thread implementation at DFSOutputStream.java
Date Tue, 03 Jan 2017 17:47:52 GMT
Hi Shu,

Apologies for dealy in response.

Close packet is something which is the last packet of the current block
after writing which , the current block is closed and a new pipeline is
setup for the next block.

Your understanding of the one.isLastPacketInBlock() is correct.


On Wed, Dec 14, 2016 at 9:37 PM, wang shu <uchicagoshuwang@gmail.com> wrote:

> Hi all,
> I have a question regarding the streamer thread (Line 382-575)
>  implementation at DFSOutputStream.java(http://gr
> epcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/had
> oop-hdfs/2.7.1/org/apache/hadoop/hdfs/DFSOutputStream.java#
> DFSOutputStream.DataStreamer.run%28%29).
> What's the logic of line 468-485 and line 535-548, why it check the packet
> is the last one twice and the comments are different, one is "
> // wait for all data packets have been successfully acked" and another is
> "// wait for the close packet has been acked".  What's the close packet?
> My understanding of this is:
> If a packet is not the last in block, then the packet is sent directly,
> and no need to wait for ACK.
> If a packet is the last one, then it will first wait for all previous
> packet to be ACKed, and then it will send the last packet in this block(
> called close packet, I guess?), and then will need wait the last packet to
> be ACKed.
> Thanks a lot for your help.
> Best,
> Shu

View raw message