hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8391) NN should consider current EC tasks handling count from DN while assigning new tasks
Date Wed, 13 May 2015 10:47:59 GMT
Uma Maheswara Rao G created HDFS-8391:
-----------------------------------------

             Summary: NN should consider current EC tasks handling count from DN while assigning
new tasks
                 Key: HDFS-8391
                 URL: https://issues.apache.org/jira/browse/HDFS-8391
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: datanode, namenode
            Reporter: Uma Maheswara Rao G
            Assignee: Uma Maheswara Rao G


Currently NN will pick the (maxStreams-XmitsCount) number of ECtasks at a time for assigning
to the respective DN.
{code}
//get datanode commands
      final int maxTransfer = blockManager.getMaxReplicationStreams()
          - xmitsInProgress;
{code}

But right now we increment xmitsInProgress count at DN only for regular replication tasks
but not for ECWorker tasks.

So, either we should treat this logic separately for EC or we should consider EC current task
handling count form DN.

This jira for discussing more for this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message