hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5907) Improve getSplits() performance for fs implementations that can utilize performance gains from recursive listing
Date Thu, 29 May 2014 09:46:03 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14012238#comment-14012238
] 

Steve Loughran commented on MAPREDUCE-5907:
-------------------------------------------

# ..provide a joint patch with both changes in
# presumably this patch is only going to benefit object stores where the cost of any ls operation
is high due to the need to set up an HTTP link for each directory
# How tangible are the benefits for S3N? -in cluster and remote? 
# Have you implemented the same patch for the Swift object store? What were the results?


> Improve getSplits() performance for fs implementations that can utilize performance gains
from recursive listing
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5907
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5907
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client
>    Affects Versions: 2.4.0
>            Reporter: Sumit Kumar
>            Assignee: Sumit Kumar
>         Attachments: MAPREDUCE-5907.patch
>
>
> This patch utilizes benefits of recursive listing proposed in https://issues.apache.org/jira/browse/HADOOP-10634



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message