hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suhas Gogate (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-678) distcp -m option does not work
Date Mon, 29 Jun 2009 21:08:47 GMT
distcp -m option does not work 
-------------------------------

                 Key: MAPREDUCE-678
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-678
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: distcp
    Affects Versions: 0.20.1
            Reporter: Suhas Gogate


In spite of -m 10 , number of launched map tasks are 15.  

bin/hadoop  distcp -Dmapred.job.queue.name=xxxx -i -p -update -m 10   -delete  hftp://xxxx.yyyy.com:50070/user/xxxx/QA_incremental
hdfs://aaaa.yyyy.com:8020/user/xxxxx/QA_incremental'

09/06/29 19:14:24 INFO tools.DistCp: srcPaths=[hftp://xxxxx.yyyy.com:50070/user/xxxx/QA_incremental]
09/06/29 19:14:24 INFO tools.DistCp: destPath=hdfs://zzzz.yyyy.com:8020/user/xxxx/QA_incremental
09/06/29 19:14:34 INFO tools.DistCp: sourcePathsCount=62
09/06/29 19:14:34 INFO tools.DistCp: filesToCopyCount=15
09/06/29 19:14:34 INFO tools.DistCp: bytesToCopyCount=18.0g
09/06/29 19:14:37 INFO mapred.JobClient: Running job: job_200906290541_1523
09/06/29 19:14:38 INFO mapred.JobClient:  map 0% reduce 0%
09/06/29 19:15:22 INFO mapred.JobClient:  map 2% reduce 0%
09/06/29 19:26:57 INFO mapred.JobClient:  map 9% reduce 0%
09/06/29 19:36:02 INFO mapred.JobClient:  map 16% reduce 0%
09/06/29 19:36:46 INFO mapred.JobClient:  map 22% reduce 0%
09/06/29 19:37:25 INFO mapred.JobClient:  map 29% reduce 0%
09/06/29 19:38:28 INFO mapred.JobClient:  map 36% reduce 0%
09/06/29 19:39:41 INFO mapred.JobClient:  map 42% reduce 0%
09/06/29 19:40:08 INFO mapred.JobClient:  map 46% reduce 0%
09/06/29 19:44:08 INFO mapred.JobClient:  map 53% reduce 0%
09/06/29 19:44:19 INFO mapred.JobClient:  map 60% reduce 0%
09/06/29 19:44:39 INFO mapred.JobClient:  map 66% reduce 0%
09/06/29 19:45:04 INFO mapred.JobClient:  map 73% reduce 0%
09/06/29 19:46:08 INFO mapred.JobClient:  map 80% reduce 0%
09/06/29 19:46:24 INFO mapred.JobClient:  map 86% reduce 0%
09/06/29 19:46:38 INFO mapred.JobClient:  map 93% reduce 0%
09/06/29 19:48:00 INFO mapred.JobClient:  map 100% reduce 0%
09/06/29 19:48:19 INFO mapred.JobClient: Job complete: job_200906290541_1523
09/06/29 19:48:19 INFO mapred.JobClient: Counters: 10
09/06/29 19:48:19 INFO mapred.JobClient:   Job Counters 
09/06/29 19:48:19 INFO mapred.JobClient:     Launched map tasks=15
09/06/29 19:48:19 INFO mapred.JobClient:   FileSystemCounters
09/06/29 19:48:19 INFO mapred.JobClient:     HDFS_BYTES_READ=7742
09/06/29 19:48:19 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=19294982191
09/06/29 19:48:19 INFO mapred.JobClient:   distcp
09/06/29 19:48:19 INFO mapred.JobClient:     Files copied=15
09/06/29 19:48:19 INFO mapred.JobClient:     Bytes copied=19294982191
09/06/29 19:48:19 INFO mapred.JobClient:     Bytes expected=19294982191
09/06/29 19:48:19 INFO mapred.JobClient:   Map-Reduce Framework
09/06/29 19:48:19 INFO mapred.JobClient:     Map input records=16
09/06/29 19:48:19 INFO mapred.JobClient:     Spilled Records=0
09/06/29 19:48:19 INFO mapred.JobClient:     Map input bytes=3190
09/06/29 19:48:19 INFO mapred.JobClient:     Map output records=0



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message