hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6806) Trim configuration values in gridmix/DistributedCacheEmulator.java
Date Tue, 01 Nov 2016 05:55:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15624470#comment-15624470
] 

Hadoop QA commented on MAPREDUCE-6806:
--------------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s {color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red}
The patch doesn't appear to include any new or modified tests. Please justify why no new tests
are needed for this patch. Also please list what manual steps were performed to verify this
patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 17s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 13s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 17s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 28s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 18s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s {color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 11s {color} | {color:red}
hadoop-tools/hadoop-gridmix: The patch generated 2 new + 21 unchanged - 0 fixed = 23 total
(was 21) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 22s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 34s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m 57s {color} | {color:green}
hadoop-gridmix in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 18s {color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 28s {color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12836284/MAPREDUCE-6806.0000.patch
|
| JIRA Issue | MAPREDUCE-6806 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  findbugs
 checkstyle  |
| uname | Linux 122f46f5651b 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 7ba74be |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6792/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-gridmix.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6792/testReport/
|
| modules | C: hadoop-tools/hadoop-gridmix U: hadoop-tools/hadoop-gridmix |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6792/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Trim configuration values in gridmix/DistributedCacheEmulator.java
> ------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6806
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6806
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/gridmix
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Tianyin Xu
>            Priority: Minor
>         Attachments: MAPREDUCE-6806.0000.patch
>
>
> The current implementation of {{DistributedCacheEmulator.java}} in {{gridmix}} does not
follow the practice of trimming configuration values. This leads to errors if users set values
containing space or newline.
> see the previous commits as reference (just list a few):
> HADOOP-6578. Configuration should trim whitespace around a lot of value types
> HADOOP-6534. Trim whitespace from directory lists initializing
> Patch is available against trunk
> HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
> HDFS-2799. Trim fs.checkpoint.dir values.
> YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
> YARN-2869. CapacityScheduler should trim sub queue names when parse configuration.
> Patch is available against trunk (tested):
> {code:title=DistributedCacheEmulator.java|borderStyle=solid}
> index 72027c1..eb12723 100644
> --- a/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/DistributedCacheEmulator.java
> +++ b/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/DistributedCacheEmulator.java
> @@ -315,14 +315,14 @@ void updateHDFSDistCacheFilesList(JobStory jobdesc) throws IOException
{
>      // paths, to be used by this simulated job.
>      JobConf jobConf = jobdesc.getJobConf();
> -    String[] files = jobConf.getStrings(MRJobConfig.CACHE_FILES);
> +    String[] files = jobConf.getTrimmedStrings(MRJobConfig.CACHE_FILES);
>      if (files != null) {
> -      String[] fileSizes = jobConf.getStrings(MRJobConfig.CACHE_FILES_SIZES);
> +      String[] fileSizes = jobConf.getTrimmedStrings(MRJobConfig.CACHE_FILES_SIZES);
>        String[] visibilities =
> -        jobConf.getStrings(MRJobConfig.CACHE_FILE_VISIBILITIES);
> +        jobConf.getTrimmedStrings(MRJobConfig.CACHE_FILE_VISIBILITIES);
>        String[] timeStamps =
> -        jobConf.getStrings(MRJobConfig.CACHE_FILE_TIMESTAMPS);
> +        jobConf.getTrimmedStrings(MRJobConfig.CACHE_FILE_TIMESTAMPS);
>        FileSystem fs = FileSystem.get(conf);
>        String user = jobConf.getUser();
> @@ -495,7 +495,7 @@ void configureDistCacheFiles(Configuration conf, JobConf jobConf)
>        throws IOException {
>      if (shouldEmulateDistCacheLoad()) {
> -      String[] files = jobConf.getStrings(MRJobConfig.CACHE_FILES);
> +      String[] files = jobConf.getTrimmedStrings(MRJobConfig.CACHE_FILES);
>        if (files != null) {
>          // hdfs based distributed cache files to be configured for simulated job
>          List<String> cacheFiles = new ArrayList<String>();
> @@ -504,10 +504,10 @@ void configureDistCacheFiles(Configuration conf, JobConf jobConf)
>          List<String> localCacheFiles = new ArrayList<String>();
>          String[] visibilities =
> -          jobConf.getStrings(MRJobConfig.CACHE_FILE_VISIBILITIES);
> +          jobConf.getTrimmedStrings(MRJobConfig.CACHE_FILE_VISIBILITIES);
>          String[] timeStamps =
> -          jobConf.getStrings(MRJobConfig.CACHE_FILE_TIMESTAMPS);
> -        String[] fileSizes = jobConf.getStrings(MRJobConfig.CACHE_FILES_SIZES);
> +          jobConf.getTrimmedStrings(MRJobConfig.CACHE_FILE_TIMESTAMPS);
> +        String[] fileSizes = jobConf.getTrimmedStrings(MRJobConfig.CACHE_FILES_SIZES);
>          String user = jobConf.getUser();
>          for (int i = 0; i < files.length; i++) {
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message