hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "McWhorter, David" <David_McWhor...@PremierInc.com>
Subject Re: connection pooling for hive JDBC client
Date Mon, 08 Jun 2015 15:01:00 GMT
Thank you Thejas, this is helpful.
— 

David McWhorter
Senior Developer, Foundations
Informatics and Technology Services
Office: 434.260.5232 | Mobile: 434.227.2551
david_mcwhorter@premierinc.com  |  Premier, Inc. (NASDAQ: PINC)










On 6/5/15, 12:14 PM, "Thejas Nair" <thejas.nair@gmail.com> wrote:

>It is better to use re-use the hive jdbc connection if possible, as
>there are costs associated with creating new connection. However,
>there are known issues in running multiple queries simultaneously
>using a single connection. For now, it woudl be safer to run queries
>sequentially within a connection (ie, don't share it across threads
>and run concurrent queries). (You can ofcourse run queries in parallel
>using different connections).
>
>Regarding connection pooling libraries, I am not sure if that has been
>tested. There might be issues like that unsupported api call you
>mentioned, that need to be fixed.
>
>
>
>On Fri, Jun 5, 2015 at 6:50 AM, McWhorter, David
><David_McWhorter@premierinc.com> wrote:
>> Hello, I sent this email to the users list a few days ago but no one there seems
to be able to help, so I am trying the dev list.  I am working on an application that queries
and interacts with hive using the JDBC API.  In many other cases, using a JDBC connection
pool such as commons-dbcp or BoneCP or HikariCP is a recommended practice and results in much
better performance.  All of the examples I’ve found of using accessing Hive through the
JDBC API (such as https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBCClientSampleCode)
use a raw JDBC connection to Hive directly.  A few questions:
>>
>>   1.  Should hive work with connection pools such as commons-dbcp, BoneCP, or HikariCP?
>>      *   Note: I have tried all BoneCP and HikariCP, and cannot seem to get either
to work because HiveConnection.setReadOnly throws a java.sql.SQLException saying “Method
not supported”.  I am using hive 0.14.0.
>>   2.  Do you recommended using a JDBC connection pool for interacting with hive from
an application that will execute many repeated and concurrent queries/statements?
>>
>> Also, just to be clear, I am not asking about configuring the Hive metastore to use
a connection pool to connect to its underlying database (MySQL, PostgreSQL, etc), but about
using a connection pool to interact with and query Hive through the JDBC api from an application.
>>
>> Thank you,
>> David McWhorter
>>
>> —
>>
>> David McWhorter
>> Senior Developer, Foundations
>> Informatics and Technology Services
>> Office: 434.260.5232 | Mobile: 434.227.2551
>> david_mcwhorter@premierinc.com<mailto:david_mcwhorter@premierinc.com>  |  Premier,
Inc. (NASDAQ: PINC)
Mime
View raw message