hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (MAPREDUCE-6877) Assign map task preferentially to the data node where the split is on faster storage type
Date Tue, 18 Apr 2017 09:01:41 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kai Zheng reassigned MAPREDUCE-6877:
------------------------------------

    Assignee: Tim Yao

> Assign map task preferentially to the data node where the split is on faster storage
type
> -----------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6877
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6877
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Tim Yao
>            Assignee: Tim Yao
>
> It would be good to use SSD in HDFS to improve reading/writing performance. However,
SSD costs more than HDD, so there is a tradeoff policy ONE-SSD to balance the performance
and cost. But there occurs a problem whether applications will read the replication on SSD
or not. If applications wouldn’t preferentially read the replication on SSD, the advantage
of SSD wouldn’t be fully utilized. The current MapReduce only assign tasks according to
data locality. The storage types of all the replications of each split should also be taken
into consideration in order to assign map task preferentially to a node where its split is
located on a faster storage type.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message