hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Mace (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-5445) PacketReceiver populates PacketHeader dataPlusChecksumLen incorrectly
Date Tue, 29 Oct 2013 22:51:25 GMT
Jonathan Mace created HDFS-5445:
-----------------------------------

             Summary: PacketReceiver populates PacketHeader dataPlusChecksumLen incorrectly
                 Key: HDFS-5445
                 URL: https://issues.apache.org/jira/browse/HDFS-5445
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: datanode
    Affects Versions: 2.2.0, 2.1.0-beta
         Environment: Ubuntu 12.10, Hadoop 2.1.0-beta
            Reporter: Jonathan Mace
            Priority: Minor


Summary: PacketReceiver reconstructs PacketHeaders with a packetLen 4 bytes fewer than it
should be.  It doesn't cause any exceptions because the reconstructed header is never reserialized,
and the packetLen field is not used in this part of the code.

In the BlockSender class, when a Packet is constructed it must be passed the field packetLen,
which is defined as the data length, checksum data length, PLUS the length of the packetLen
field itself (4 byte integer).

{code:title=BlockSender.java|borderStyle=solid}
484:  private int sendPacket(ByteBuffer pkt, int maxChunks, OutputStream out,
485:      boolean transferTo, DataTransferThrottler throttler) throws IOException {
...
491:    int packetLen = dataLen + checksumDataLen + 4;
...
504:    int headerLen = writePacketHeader(pkt, dataLen, packetLen);
...
586:  }
...
792:  private int writePacketHeader(ByteBuffer pkt, int dataLen, int packetLen) {
793:    pkt.clear();
794:    // both syncBlock and syncPacket are false
795:    PacketHeader header = new PacketHeader(packetLen, offset, seqno,
796:        (dataLen == 0), dataLen, false);
...
802:  }
{code}

In the PacketReceiver class, the PacketHeader is reconstructed using the method setFieldsFromData.
 However, the 4 bytes for the packetLen field length are missing.

{code:title=PacketReceiver.java|borderStyle=solid}
112:  private void doRead(ReadableByteChannel ch, InputStream in)
113:      throws IOException {
...
136:    int payloadLen = curPacketBuf.getInt();
...
144:    int dataPlusChecksumLen = payloadLen - Ints.BYTES;
...
181:    curHeader.setFieldsFromData(dataPlusChecksumLen, headerBuf);
...
192:  }
{code}

I found this because I was making small modifications to the code that exposed this inconsistency.






--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message