hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7878) Regression HADOOP-7777 switch changes too clever for the StaticMapping
Date Sun, 11 Dec 2011 18:10:40 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167154#comment-13167154
] 

Steve Loughran commented on HADOOP-7878:
----------------------------------------

No javadoc warnings are being generated. This patch only touches test/ files that aren't included
in the javadoc targets. Local diffs of before/after only show changes in download times/bandwidth

$ cat patchJavadocWarnings.txt 
[INFO] Scanning for projects...
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for org.apache.hadoop:hadoop-common:jar:0.24.0-SNAPSHOT
[WARNING] 'dependencyManagement.dependencies.dependency.(groupId:artifactId:type:classifier)'
must be unique: org.apache.hadoop:hadoop-hdfs:test-jar -> duplicate declaration of version
${project.version} @ org.apache.hadoop:hadoop-project:0.24.0-SNAPSHOT, /Users/slo/Java/Hadoop/github/trunk/hadoop-project/pom.xml,
line 140, column 19
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they threaten the stability
of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support building such malformed
projects.
[WARNING] 
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Hadoop Common 0.24.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-annotations/0.24.0-SNAPSHOT/maven-metadata.xml
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-annotations/0.24.0-SNAPSHOT/maven-metadata.xml
(797 B at 0.3 KB/sec)
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-annotations/0.24.0-SNAPSHOT/hadoop-annotations-0.24.0-20111210.183528-1162.pom
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-annotations/0.24.0-SNAPSHOT/hadoop-annotations-0.24.0-20111210.183528-1162.pom
(2 KB at 2.9 KB/sec)
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-project/0.24.0-SNAPSHOT/maven-metadata.xml
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-project/0.24.0-SNAPSHOT/maven-metadata.xml
(828 B at 2.0 KB/sec)
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-project/0.24.0-SNAPSHOT/hadoop-project-0.24.0-20111210.183201-566.pom
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-project/0.24.0-SNAPSHOT/hadoop-project-0.24.0-20111210.183201-566.pom
(24 KB at 23.4 KB/sec)
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-main/0.24.0-SNAPSHOT/maven-metadata.xml
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-main/0.24.0-SNAPSHOT/maven-metadata.xml
(611 B at 1.6 KB/sec)
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-main/0.24.0-SNAPSHOT/hadoop-main-0.24.0-20111210.183200-306.pom
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-main/0.24.0-SNAPSHOT/hadoop-main-0.24.0-20111210.183200-306.pom
(13 KB at 20.5 KB/sec)
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-auth/0.24.0-SNAPSHOT/maven-metadata.xml
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-auth/0.24.0-SNAPSHOT/maven-metadata.xml
(1008 B at 2.6 KB/sec)
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-auth/0.24.0-SNAPSHOT/hadoop-auth-0.24.0-20111210.183530-1134.pom
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-auth/0.24.0-SNAPSHOT/hadoop-auth-0.24.0-20111210.183530-1134.pom
(7 KB at 9.8 KB/sec)
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-annotations/0.24.0-SNAPSHOT/hadoop-annotations-0.24.0-20111210.183528-1162.jar
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-auth/0.24.0-SNAPSHOT/hadoop-auth-0.24.0-20111210.183530-1134.jar
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-annotations/0.24.0-SNAPSHOT/hadoop-annotations-0.24.0-20111210.183528-1162.jar
(17 KB at 18.2 KB/sec)
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-auth/0.24.0-SNAPSHOT/hadoop-auth-0.24.0-20111210.183530-1134.jar
(42 KB at 26.8 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ hadoop-common ---
[INFO] Deleting /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-common ---
[INFO] Executing tasks

main:
    [mkdir] Created dir: /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/test-dir
    [mkdir] Created dir: /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/test/data
[INFO] Executed tasks
[INFO] 
[INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @ hadoop-common ---
[INFO] Source directory: /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/generated-sources/java
added.
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (compile-proto) @ hadoop-common ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (save-version) @ hadoop-common ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @ hadoop-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ hadoop-common ---
[INFO] Compiling 583 source files to /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/classes
[INFO] 
[INFO] --- build-helper-maven-plugin:1.5:add-test-source (add-test-source) @ hadoop-common
---
[INFO] Test Source directory: /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java
added.
[INFO] 
[INFO] --- avro-maven-plugin:1.5.3:schema (generate-avro-test-sources) @ hadoop-common ---
[INFO] 
[INFO] --- avro-maven-plugin:1.5.3:protocol (generate-avro-test-sources) @ hadoop-common ---
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (compile-test-proto) @ hadoop-common ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (generate-test-sources) @ hadoop-common ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) @ hadoop-common
---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 11 resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (create-log-dir) @ hadoop-common ---
[INFO] Executing tasks

main:
   [delete] Deleting directory /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/test/data
    [mkdir] Created dir: /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/test/data
    [mkdir] Created dir: /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/log
     [copy] Copying 4 files to /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/test-classes
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ hadoop-common ---
[INFO] Compiling 276 source files to /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:2.10:test (default-test) @ hadoop-common ---
[INFO] Tests are skipped.
[INFO] 
[INFO] >>> maven-javadoc-plugin:2.7:javadoc (default-cli) @ hadoop-common >>>
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-common ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @ hadoop-common ---
[INFO] Source directory: /Users/slo/Java/Hadoop/github/trunk/hadoop-common-project/hadoop-common/target/generated-sources/java
added.
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (compile-proto) @ hadoop-common ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (save-version) @ hadoop-common ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] <<< maven-javadoc-plugin:2.7:javadoc (default-cli) @ hadoop-common <<<
[INFO] 
[INFO] --- maven-javadoc-plugin:2.7:javadoc (default-cli) @ hadoop-common ---
[INFO] 
ExcludePrivateAnnotationsStandardDoclet
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:17.934s
[INFO] Finished at: Sun Dec 11 17:40:12 GMT 2011
[INFO] Final Memory: 24M/114M
[INFO] ------------------------------------------------------------------------
                
> Regression HADOOP-7777 switch changes too clever for the StaticMapping
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-7878
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7878
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: util
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>              Labels: regression
>         Attachments: HADOOP-7878.patch, HADOOP-7878.patch
>
>   Original Estimate: 0.5h
>  Remaining Estimate: 0.5h
>
> This doesn't show up until you apply the HDFS-2492 patch, but the attempt to make the
{{StaticMapping}} topology clever by deciding if it is single rack or multi rack based on
its rack->node mapping breaks the HDFS {{TestBlocksWithNotEnoughRacks}} test. Why? Because
the racks go in after the switch topology is cached by the {{BlockManager}}, which assumes
the system is always single-switch.
> Fix: default to assuming multi-switch; remove the intelligence, add a setter for anyone
who really wants to simulate single-switch racks. 
> Test: verify that a newly created simple mapping is multi switch

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message