hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Takanobu Asanuma <tasan...@yahoo-corp.jp>
Subject Re: Review Request 40867: HIVE-11527 - bypass HiveServer2 thrift interface for query results
Date Sat, 19 Mar 2016 01:15:43 GMT


> On 3月 18, 2016, 5:57 p.m., Sergey Shelukhin wrote:
> >

Thanks for the reviewing.


> On 3月 18, 2016, 5:57 p.m., Sergey Shelukhin wrote:
> > jdbc/src/java/org/apache/hive/jdbc/HiveQueryResultSet.java, line 210
> > <https://reviews.apache.org/r/40867/diff/6-7/?file=1230821#file1230821line210>
> >
> >     nit: whitespace

Sorry, I'll fix it.


> On 3月 18, 2016, 5:57 p.m., Sergey Shelukhin wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/Driver.java, line 1919
> > <https://reviews.apache.org/r/40867/diff/6-7/?file=1230823#file1230823line1919>
> >
> >     why cannot the conf itself be passed in? why is the map necessary?
> >     Or, if that doesn't work for some reason, what about creating a small struct
class to hold the fields with logical names and types, instead of the map.

> why cannot the conf itself be passed in?

In this patch, HiveServer2 gives JDBC clients the configurations which is the necessary and
sufficient condition for resolving namespace of HA.
In present implementation of Hive, the configurations on the JDBC clients side are different
from ones on the cluster side. So if all the configurations are passed in here, it would destroy
compatibility with existing JDBC applications. Even if it is better, I think it should be
done in another jira.

> what about creating a small struct class to hold the fields with logical names and types,
instead of the map.

We need a simple key-value structure in here. Map is a simple way to do this and we don't
need to define the new structure in the thrift API. So I used the map. What the advantages
of creating a new structure class?


- Takanobu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40867/#review124234
-----------------------------------------------------------


On 3月 16, 2016, 8:51 a.m., Takanobu Asanuma wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40867/
> -----------------------------------------------------------
> 
> (Updated 3月 16, 2016, 8:51 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> This is a WIP patch for HIVE-11527
> 
> * I added a new configuration whose name is hive.server2.webhdfs.bypass.enabled. The
default is false. When this value is true, clients use the bypass.
> 
> * I still have not considered security such as Kerberos and SSL at present.
> 
> * I have not implement Statement#setFetchSize for bypass yet.
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 98c6372 
>   jdbc/src/java/org/apache/hive/jdbc/HiveQueryResultSet.java 8f67209 
>   jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java b4dba44 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java 7327a42 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java 0b0c336 
>   service-rpc/if/TCLIService.thrift aa28b6e 
>   service-rpc/src/gen/thrift/gen-cpp/TCLIService_types.h 7f1d9dd 
>   service-rpc/src/gen/thrift/gen-cpp/TCLIService_types.cpp 3a27a60 
>   service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TColumnDesc.java
31472c8 
>   service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TExecuteStatementResp.java
7101fa5 
>   service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TGetTablesReq.java
1aa3f94 
>   service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TProtocolVersion.java
14d50ed 
>   service-rpc/src/gen/thrift/gen-php/Types.php b7df50a 
>   service-rpc/src/gen/thrift/gen-py/TCLIService/ttypes.py c691781 
>   service-rpc/src/gen/thrift/gen-py/__init__.py PRE-CREATION 
>   service-rpc/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb 07ed97c 
>   service/src/java/org/apache/hive/service/cli/CLIService.java ab30ae2 
>   service/src/java/org/apache/hive/service/cli/ColumnDescriptor.java 7bd9f06 
>   service/src/java/org/apache/hive/service/cli/operation/Operation.java d9a273b 
>   service/src/java/org/apache/hive/service/cli/operation/OperationManager.java 56a9c18

>   service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java 04d816a 
>   service/src/java/org/apache/hive/service/cli/session/HiveSession.java 4f4e92d 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 8baecdf 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 62fcde5 
> 
> Diff: https://reviews.apache.org/r/40867/diff/
> 
> 
> Testing
> -------
> 
> I have tested few simple queries and they worked well. But I think there are some problems
for some queries. I'm going to test more queries and fix bugs. I'm also going to add unit
tests.
> 
> 
> Thanks,
> 
> Takanobu Asanuma
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message