Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Sun, 9 Sep 2012 01:00:21 +1100 (NCT)
From: "Hudson (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <845460390.54248.1347112821541.JavaMail.jiratomcat@arcas>
In-Reply-To: <558156197.46357.1346960347717.JavaMail.jiratomcat@arcas>
Subject: [jira] [Commented] (HADOOP-8775) MR2 distcp permits non-positive
 value to -bandwidth option which causes job never to complete
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451338#comment-13451338 ] 

Hudson commented on HADOOP-8775:
--------------------------------

Integrated in Hadoop-Mapreduce-trunk #1190 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1190/])
    HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete. Contributed by Sandy Ryza. (Revision 1382119)

     Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382119
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java

                
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-8775
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8775
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>             Fix For: 2.0.2-alpha
>
>         Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job:  map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job:  map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job:  map 100% reduce 0%
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira