hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gil Vernik (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-6857) Reduce number of exists() calls on the target object
Date Mon, 06 Mar 2017 11:25:32 GMT
Gil Vernik created MAPREDUCE-6857:

             Summary: Reduce number of exists() calls on the target object
                 Key: MAPREDUCE-6857
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6857
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: distcp
    Affects Versions: 3.0.0-alpha2
            Reporter: Gil Vernik
             Fix For: 3.0.0-alpha2

CopyMapper.map(..) calls targetStatus = targetFS.getFileStatus(target).
Few steps later RetriableFileCopyCommand.promoteTmpToTarget(..) will call again exists(target)
and delete if present. 
The second exists() is useless, since if targetStatus==null it can be easily seen  if overwrite
mode is activated and so target object can be deleted.

The propose of this patch is to delete target object by using targetStatus and thus avoid
calling exists() method.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org

View raw message