Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Tue, 4 Mar 2014 20:08:21 +0000 (UTC)
From: "Eric Hanson (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12698715.1393963505404.13181.1393963701322@arcas>
In-Reply-To: <JIRA.12698715.1393963505404@arcas>
References: <JIRA.12698715.1393963505404@arcas>
Subject: [jira] [Updated] (HIVE-6546) WebHCat job submission for pig with
 -useHCatalog argument fails on Windows
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HIVE-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Hanson updated HIVE-6546:
------------------------------

    Environment: 
HDInsight deploying HDP 1.3:  c:\apps\dist\pig-0.11.0.1.3.2.0-05
Also on Windows HDP 1.3 one-box configuration.

  was:Windows Azure HDINSIGHT and Windows one-box installations.


> WebHCat job submission for pig with -useHCatalog argument fails on Windows
> --------------------------------------------------------------------------
>
>                 Key: HIVE-6546
>                 URL: https://issues.apache.org/jira/browse/HIVE-6546
>             Project: Hive
>          Issue Type: Bug
>          Components: WebHCat
>    Affects Versions: 0.11.0, 0.12.0, 0.13.0
>         Environment: HDInsight deploying HDP 1.3:  c:\apps\dist\pig-0.11.0.1.3.2.0-05
> Also on Windows HDP 1.3 one-box configuration.
>            Reporter: Eric Hanson
>
> On a one-box windows setup, do the following from a powershell prompt:
> cmd /c curl.exe -s `
>   -d user.name=hadoop `
>   -d arg=-useHCatalog `
>   -d execute="emp = load '/data/emp/emp_0.dat'; dump emp;" `
>   -d statusdir="/tmp/webhcat.output01" `
>   'http://localhost:50111/templeton/v1/pig' -v
> The job fails with error code 7, but it should run. 
> I traced this down to the following. In the job configuration for the TempletonJobController, we have templeton.args set to
> cmd,/c,call,C:\\hadoop\\\\pig-0.11.0.1.3.0.0-0846/bin/pig.cmd,-D__WEBHCAT_TOKEN_FILE_LOCATION__="-useHCatalog",-execute,"emp = load '/data/emp/emp_0.dat'; dump emp;"
> Notice the = sign before "-useHCatalog". I think this should be a comma.
> The bad string D__WEBHCAT_TOKEN_FILE_LOCATION__="-useHCatalog" gets created in  org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows().
> It happens at line 434:
> {code}
>       } else {
>           if (i < args.length - 1) {
>             prop += "=" + args[++i];   // RIGHT HERE! at iterations i = 37, 38
>           }
>         }
> {code}
> Bug is here:
> {code}
>       if (prop != null) {
>         if (prop.contains("=")) {  // -D__WEBHCAT_TOKEN_FILE_LOCATION__ does not contain equal, so else branch is run and appends ="-useHCatalog",
>           // everything good
>         } else {
>           if (i < args.length - 1) {
>             prop += "=" + args[++i];
>           }
>         }
>         newArgs.add(prop);
>       }
> {code}
> One possible fix is to change the string constant org.apache.hcatalog.templeton.tool.TempletonControllerJob.TOKEN_FILE_ARG_PLACEHOLDER to have an "=" sign in it. Or, preProcessForWindows() itself could be changed.


--
This message was sent by Atlassian JIRA
(v6.2#6252)