hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dragon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-10059) CLONE - Refactor DFSStripedOutputStream and StripedDataStreamer
Date Tue, 15 Mar 2016 08:16:35 GMT
dragon created HDFS-10059:

             Summary: CLONE - Refactor DFSStripedOutputStream and StripedDataStreamer
                 Key: HDFS-10059
                 URL: https://issues.apache.org/jira/browse/HDFS-10059
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: hdfs-client
            Reporter: dragon
            Assignee: Tsz Wo Nicholas Sze

- DFSStripedOutputStream has a list of StripedDataStreamer(s).  The streamers share a data
structure List<BlockingQueue<LocatedBlock>> stripeBlocks for communicate located
block and end block information.
For example,
      // before retrieving a new block, transfer the finished block to
      // leading streamer
      LocatedBlock finishedBlock = new LocatedBlock(
          new ExtendedBlock(block.getBlockPoolId(), block.getBlockId(),
              block.getNumBytes(), block.getGenerationStamp()), null);
      try {
        boolean offSuccess = stripedBlocks.get(0).offer(finishedBlock, 30,
It is unnecessary to create a LocatedBlock object for an end block since the locations passed
is null.  Also, the return value is ignored (i.e. offSuccess is not used).

- DFSStripedOutputStream has another data structure cellBuffers for computing parity.  It
should be refactored to a class.

This message was sent by Atlassian JIRA

View raw message