hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rakesh R (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9879) Erasure Coding : schedule striped blocks to be cached on DataNodes
Date Tue, 01 Mar 2016 05:59:18 GMT
Rakesh R created HDFS-9879:

             Summary: Erasure Coding : schedule striped blocks to be cached on DataNodes
                 Key: HDFS-9879
                 URL: https://issues.apache.org/jira/browse/HDFS-9879
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Rakesh R
            Assignee: Rakesh R

This jira to discuss and implement the caching of striped block objects on the appropriate

Presently it is checking block group size and scheduling the blockGroupId to the datanode,
which needs to be refined by checking the {{StripedBlockUtil.getInternalBlockLength()}} and
schedule proper blockId to the datanode.

      if (pendingCapacity < blockInfo.getNumBytes()) {
        LOG.trace("Block {}: DataNode {} is not a valid possibility " +
            "because the block has size {}, but the DataNode only has {} " +
            "bytes of cache remaining ({} pending bytes, {} already cached.)",
            blockInfo.getBlockId(), datanode.getDatanodeUuid(),
            blockInfo.getNumBytes(), pendingCapacity, pendingBytes,

    for (DatanodeDescriptor datanode : chosen) {
      LOG.trace("Block {}: added to PENDING_CACHED on DataNode {}",
          blockInfo.getBlockId(), datanode.getDatanodeUuid());
      boolean added = datanode.getPendingCached().add(cachedBlock);
      assert added;

This message was sent by Atlassian JIRA

View raw message