hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carl Steinbach (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2935) Implement HiveServer2
Date Thu, 04 Oct 2012 17:27:50 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469526#comment-13469526
] 

Carl Steinbach commented on HIVE-2935:
--------------------------------------

The attached patches contain a complete, working version of HiveServer2. Before going into
details about the contents of the patch I want to first quickly review how to apply the patch
and try out the new server:

1) Download and apply HIVE-2935.2.nothrift.patch.txt
2) Run the Thrift code generator (make sure $THRIFT_HOME points to thrift version 0.7.0)
{{% ant thriftif -Dthrift.home=$THRIFT_HOME}}
3) Optionally download and unpack the beeline test outputs in the ql/src/test/results directory.

4) Build Hive:
{{% ant clean package}}
5) Start HiveServer2
{{% hiveserver2}}
6) From another window start the beeline CLI and connect to HiveServer2:
{noformat}
% beeline
Hive version 0.10.0-SNAPSHOT by Apache
beeline> !connect jdbc:hive2://localhost:10000 scott tiger org.apache.hive.jdbc.HiveDriver
!connect jdbc:hive2://localhost:10000 scott tiger org.apache.hive.jdbc.HiveDriver
Connecting to jdbc:hive2://localhost:10000
Connected to: Hive (version 0.10.0)
Driver: Hive (version 0.10.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> show tables;
show tables;
+-------------------+
|     tab_name      |
+-------------------+
| primitives        |
| src               |
| src1              |
| src_json          |
| src_sequencefile  |
| src_thrift        |
| srcbucket         |
| srcbucket2        |
| srcpart           |
+-------------------+
9 rows selected (1.079 seconds)
{noformat}

If you downloaded the test outputs you can also try running the new BeeLineTestDriver:

% ant test -Dtestcase=TestBeeLineDriver -Dtest.concurrency.num.threads=10

Patch contents:
* {{cli/src/java/org/apache/hive/cli/beeline/*.java}}
** This a JDBC CLI for Hive based on the SQLLine CLI. An earlier version of BeeLine which
had a dependency on SQLLine was previously added in HIVE-3100. However, while working on HS2
we discovered bugs in SQLLine that we needed to fix, and also needed to build in some extensions
in order to support the BeeLine test driver. Adding the code directly to Hive seemed like
the best option since the upstream project is no longer actively maintained.
* {{cli/src/java/org/apache/hive/cli/beeline/util/QFileClient.java}}
** This is a beeline test client used by TestBeeLineDriver. This class also provides an improved
version of the output masking functionality currently located in QTestUtil.
* {{common/src/java/org/apache/hive/common/util/*.java}}
** Utility classes borrowed from Hadoop.
* {{data/files/types/primitives/*}}
** Data files for a 'primitives' table that contains all Hive primitive types along with NULLs.
* {{data/scripts/q_test_*.sql}}
** Test initialization scripts that are used to create and initialize all of the tables that
are referenced by CliDriver tests. Called from QFileClient.
* {{jdbc/src/java/org/apache/hive/jdbc/*}}
** The HS2 JDBC driver.
* {{service/if/cli_service.thrift}}
** The CliService Thrift IDL file.
* {{service/src/java/org/apache/hive/service/*}}
** Service infrastructure classes borrowed from o.a.hadoop.yarn.service
* {{service/src/java/org/apache/hive/service/auth/*}}
** Kerberos/LDAP/SASL auth code for HS2
* {{service/src/java/org/apache/hive/service/cli/*}}
** Implementation classes for CLIService. These classes form the core of HiveServer2.
* {{service/src/java/org/apache/hive/service/server/HiveServer2.java}}
** HiveServer2 class. At the moment it just starts the CLIService, but we plan to extend it
with other pluggable services in the future.
* {{testutils/junit/*}
** Utility classes for running concurrent JUnit tests. Most of this code was borrowed from
tempus-fugit.

This code was a collaborative effort between me and my colleague Prasad Mujumdar. Prasad is
wholly responsible for the new JDBC driver and authorization code. We worked together on the
other parts.

We plan to break this monolithic patch up into several smaller patches in order to make the
review process easier. Here's our initial plan for how to do this:

# Review/commit the beeline CLI (note that this can be used with the existing JDBC driver
and HiveServer1)
# Review/commit the BeeLine test driver and new test outputs
# Review/commit the HiveServer2 core component.
# Review/commit the HiveServer2 JDBC driver.

We would appreciate receiving feedback from the Hive committers about whether or not this
plan makes sense.

Thanks!


                
> Implement HiveServer2
> ---------------------
>
>                 Key: HIVE-2935
>                 URL: https://issues.apache.org/jira/browse/HIVE-2935
>             Project: Hive
>          Issue Type: New Feature
>          Components: Server Infrastructure
>            Reporter: Carl Steinbach
>            Assignee: Carl Steinbach
>              Labels: HiveServer2
>         Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt,
HIVE-2935.2.nothrift.patch.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message