hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thejas Nair <thejas.n...@gmail.com>
Subject Re: connection pooling for hive JDBC client
Date Fri, 05 Jun 2015 16:14:46 GMT
It is better to use re-use the hive jdbc connection if possible, as
there are costs associated with creating new connection. However,
there are known issues in running multiple queries simultaneously
using a single connection. For now, it woudl be safer to run queries
sequentially within a connection (ie, don't share it across threads
and run concurrent queries). (You can ofcourse run queries in parallel
using different connections).

Regarding connection pooling libraries, I am not sure if that has been
tested. There might be issues like that unsupported api call you
mentioned, that need to be fixed.



On Fri, Jun 5, 2015 at 6:50 AM, McWhorter, David
<David_McWhorter@premierinc.com> wrote:
> Hello, I sent this email to the users list a few days ago but no one there seems to be
able to help, so I am trying the dev list.  I am working on an application that queries and
interacts with hive using the JDBC API.  In many other cases, using a JDBC connection pool
such as commons-dbcp or BoneCP or HikariCP is a recommended practice and results in much better
performance.  All of the examples I’ve found of using accessing Hive through the JDBC API
(such as https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBCClientSampleCode)
use a raw JDBC connection to Hive directly.  A few questions:
>
>   1.  Should hive work with connection pools such as commons-dbcp, BoneCP, or HikariCP?
>      *   Note: I have tried all BoneCP and HikariCP, and cannot seem to get either to
work because HiveConnection.setReadOnly throws a java.sql.SQLException saying “Method not
supported”.  I am using hive 0.14.0.
>   2.  Do you recommended using a JDBC connection pool for interacting with hive from
an application that will execute many repeated and concurrent queries/statements?
>
> Also, just to be clear, I am not asking about configuring the Hive metastore to use a
connection pool to connect to its underlying database (MySQL, PostgreSQL, etc), but about
using a connection pool to interact with and query Hive through the JDBC api from an application.
>
> Thank you,
> David McWhorter
>
> —
>
> David McWhorter
> Senior Developer, Foundations
> Informatics and Technology Services
> Office: 434.260.5232 | Mobile: 434.227.2551
> david_mcwhorter@premierinc.com<mailto:david_mcwhorter@premierinc.com>  |  Premier,
Inc. (NASDAQ: PINC)

Mime
View raw message