hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajith S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7947) Validate XMLs if a relevant tool is available, when using scripts
Date Fri, 08 May 2015 09:54:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14534239#comment-14534239
] 

Ajith S commented on HADOOP-7947:
---------------------------------

Thanks for the patch

I was testing the patch and have few comments/questions

1. According to the help, *-conffile* can accept a file or folder, but when we give neither,
its throwing stacktrace, its better we instead return a normal message like "Expected a File
or Folder". Just a suggestion
{quote}
*root@voltan:/opt/app/hadoop-2.7.0/bin# ./hadoop conftest -h*
Usage: hadoop conftest [-conffile <path>|-h|--help]
  Options:

  -conffile <path>
    If specified, that path will be verified. You can specify this option multiple times.
    Both file and directory are acceptable. If a directory specified,
    the files whose name end with .xml in that directory will be verified.
    If not specified, the files whose name end with .xml in $HADOOP_CONF_DIR will be verified.
  -h, --help       Print this help
*root@voltan:/opt/app/hadoop-2.7.0/bin# ./hadoop conftest -conffile*
Exception in thread "main" org.apache.commons.cli.MissingArgumentException: Missing argument
for option: conffile
        at org.apache.commons.cli.Parser.processArgs(Parser.java:343)
        at org.apache.commons.cli.Parser.processOption(Parser.java:393)
        at org.apache.commons.cli.Parser.parse(Parser.java:199)
        at org.apache.commons.cli.Parser.parse(Parser.java:85)
        at org.apache.hadoop.util.ConfTest.main(ConfTest.java:220)
{quote}


2. *conftest -conf* behaviour is quite confusing when argument is missing, its printing the
help *and* also its validating the default folder, is this fine.?? i feel in this scenario
it should have printed the error with generic usage and return.
{quote}
*root@voltan:/opt/app/hadoop-2.7.0/bin# ./hadoop conftest -conf*
15/05/08 15:03:25 WARN util.GenericOptionsParser: options parsing failed: Missing argument
for option: conf
usage: general options are:
 -archives <paths>                  comma separated archives to be
                                    unarchived on the compute machines.
 -conf <configuration file>         specify an application configuration
                                    file
 -D <property=value>                use value for given property
 -files <paths>                     comma separated files to be copied to
                                    the map reduce cluster
 -fs <local|namenode:port>          specify a namenode
 -jt <local|resourcemanager:port>   specify a ResourceManager
 -libjars <paths>                   comma separated jar files to include
                                    in the classpath.
 -tokenCacheFile <tokensFile>       name of the file with the tokens
/opt/app/hadoop-2.7.0/etc/hadoop/hdfs-site.xml: valid
/opt/app/hadoop-2.7.0/etc/hadoop/capacity-scheduler.xml: valid
/opt/app/hadoop-2.7.0/etc/hadoop/hadoop-policy.xml: valid
/opt/app/hadoop-2.7.0/etc/hadoop/core-site.xml: valid
/opt/app/hadoop-2.7.0/etc/hadoop/kms-acls.xml: valid
/opt/app/hadoop-2.7.0/etc/hadoop/kms-site.xml: valid
/opt/app/hadoop-2.7.0/etc/hadoop/mapred-site.xml: valid
/opt/app/hadoop-2.7.0/etc/hadoop/yarn-site.xml: valid
/opt/app/hadoop-2.7.0/etc/hadoop/httpfs-site.xml: valid
OK
root@voltan:/opt/app/hadoop-2.7.0/bin#
{quote}



> Validate XMLs if a relevant tool is available, when using scripts
> -----------------------------------------------------------------
>
>                 Key: HADOOP-7947
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7947
>             Project: Hadoop Common
>          Issue Type: Wish
>          Components: scripts
>    Affects Versions: 2.7.0
>            Reporter: Harsh J
>            Assignee: Kengo Seki
>              Labels: BB2015-05-TBR, newbie
>         Attachments: HADOOP-7947.001.patch, HADOOP-7947.002.patch, HADOOP-7947.003.patch,
HADOOP-7947.004.patch
>
>
> Given that we are locked down to using only XML for configuration and most of the administrators
need to manage it by themselves (unless a tool that manages for you is used), it would be
good to also validate the provided config XML (*-site.xml) files with a tool like {{xmllint}}
or maybe Xerces somehow, when running a command or (at least) when starting up daemons.
> We should use this only if a relevant tool is available, and optionally be silent if
the env. requests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message