hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2139) Enables HiveServer to accept -hiveconf option
Date Mon, 27 Jun 2011 20:51:51 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13055748#comment-13055748
] 

jiraposter@reviews.apache.org commented on HIVE-2139:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/958/#review919
-----------------------------------------------------------


I ran into some problems compiling this patch. Please verify that it builds using the ant
command 'ant clean package'.


bin/ext/hiveserver.sh
<https://reviews.apache.org/r/958/#comment1973>

    Does this mean that $HIVE_PORT takes precedence over another port specified using the
-p switch? If so then I think the reverse makes more sense.
    
    Also, in order to make the precedence explicit, I think it would be good to move this
logic to the HiveCli class, e.g. explicitly call System.getenv("HIVE_PORT") from HiveCli.



common/src/java/org/apache/hadoop/hive/common/cli/HiveCli.java
<https://reviews.apache.org/r/958/#comment1975>

    The HiveMetaStore and HiveServer imports are unnecessary.



common/src/java/org/apache/hadoop/hive/common/cli/HiveCli.java
<https://reviews.apache.org/r/958/#comment1974>

    The name of this class is likely to generate confusion. Maybe change it to CommonCliOpts,
or something else?



metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/958/#comment1976>

    We should try to avoid making the metastore dependent on ql. There's already an open ticket
(HIVE-850) that covers the task of moving SessionState to common. Looks like now may be a
good time to do this.



metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/958/#comment1977>

    cli.processHiveConf() copies all of the -hiveconf properties into the list of SystemProperties,
which I agree we want to do before initializing the logging system, but subsequently we need
to make sure that these same key/val properties are are also registered as HiveConf values
so that they have the opportunity to override values specified in hive-default.xml and hive-site.xml.
A similar trick is done for the CLI via the OptionsProcessor process_stage1() and process_stage2()
methods.



service/src/java/org/apache/hadoop/hive/service/HiveServer.java
<https://reviews.apache.org/r/958/#comment1978>

    hive.metastore.server.[min|max].threads already exists. We should add similar properties
for controlling the min/max number of threads for HiveServer.



service/src/java/org/apache/hadoop/hive/service/HiveServer.java
<https://reviews.apache.org/r/958/#comment1979>

    Need to read the -hiveconf properties into the SessionState's HiveConf.



service/src/java/org/apache/hadoop/hive/service/HiveServer.java
<https://reviews.apache.org/r/958/#comment1980>

    We should also set options.maxWorkerThreads. Looks like the default value for Thrift is
Integer.MAX_VALUE.


- Carl


On 2011-06-24 22:12:48, Patrick Hunt wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/958/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-06-24 22:12:48)
bq.  
bq.  
bq.  Review request for hive and Carl Steinbach.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  This patch updates HiveServer and HiveMetastore to add proper cli handling - similar
to that used in CliDriver (ie GnuParser).
bq.  
bq.  There's a common HiveCli class that's used by both main classes.
bq.  
bq.  I've attempted to make the cli's backward compatible with the prior command line processing.
Notice I've "deprecated" (via warnings, but the code still runs) if the old style CLI usage
is used.
bq.  
bq.  commands such as the following now work as expected:
bq.  
bq.  bin/hive --service hiveserver -t 200 -p 12000 --hiveconf hive.root.logger=DEBUG,console
bq.  
bq.  as does the following which generates usage information:
bq.  
bq.  bin/hive --service hiveserver -h
bq.  
bq.  Note: HiveMetastore as not initializing log4j, I updated the code to do similar to HiveServer
(otw the hiveconf hive.root.logger option above didn't work).
bq.  
bq.  
bq.  This addresses bug HIVE-2139.
bq.      https://issues.apache.org/jira/browse/HIVE-2139
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    bin/ext/hiveserver.sh b5edce4 
bq.    bin/ext/metastore.sh db15f6e 
bq.    common/src/java/org/apache/hadoop/hive/common/cli/HiveCli.java PRE-CREATION 
bq.    metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java bc58bd5 
bq.    service/src/java/org/apache/hadoop/hive/service/HiveServer.java ea04be9 
bq.  
bq.  Diff: https://reviews.apache.org/r/958/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  I couldn't find any tests for these changes, so I verfied the changes manually.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Patrick
bq.  
bq.



> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
>                 Key: HIVE-2139
>                 URL: https://issues.apache.org/jira/browse/HIVE-2139
>             Project: Hive
>          Issue Type: Improvement
>          Components: CLI
>         Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
>            Reporter: Kazuki Ohta
>            Assignee: Patrick Hunt
>         Attachments: HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem
to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
>         at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
>         at java.lang.Integer.parseInt(Integer.java:449)
>         at java.lang.Integer.parseInt(Integer.java:499)
>         at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2"
everytime. It's not convenient for separating the configuration between server-side and client-side.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message