impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Rahn <greg.r...@gmail.com>
Subject Re: Using comments in query for detailed monitoring
Date Sat, 17 Feb 2018 01:59:19 GMT
Unfortunately there is not a JIRA to track because the drivers are
maintained by a 3rd party and are not part of the Apache Impala project.
FWIW the version I tested was 2.5.43.1063

On Fri, Feb 16, 2018 at 4:52 PM, Sunil Parmar <sunilosunil@gmail.com> wrote:

> Thanks is there a JIRA we can track ?
>
> Sunil Parmar
>
> On Tue, Feb 13, 2018 at 3:18 PM, Greg Rahn <greg.rahn@gmail.com> wrote:
>
>> An upcoming release of the Impala driver will include a fix for not
>> removing comments when UseNativeQuery=1 is used.
>>
>> On Fri, Jan 26, 2018 at 10:29 AM, Greg Rahn <greg.rahn@gmail.com> wrote:
>>
>>> I was able to repro this as well and checking in with Simba who authors
>>> the driver.
>>>
>>>
>>> On Wed, Jan 17, 2018 at 9:03 PM, Sunil Parmar <sunilosunil@gmail.com>
>>> wrote:
>>>
>>>> Tried with case change but no luck.  From the connection log.
>>>>
>>>> Jan 17 20:33:38.163 TRACE 1 com.cloudera.jdbc.common.SConnection.nativeSQL("select
>>>> sum(1) from dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +++++
>>>> enter +++++
>>>> Jan 17 20:33:38.163 TRACE 1 com.cloudera.hivecommon.core.H
>>>> iveJDBCCommonConnection.toNativeSQL("select sum(1) from
>>>> dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +++++ enter +++++
>>>> Jan 17 20:33:38.164 TRACE 1 com.cloudera.hivecommon.dataen
>>>> gine.HiveJDBCDataEngine.prepare("select sum(1) from dwh.event_detail
>>>> where org_group_id > 10; "): +++++ enter +++++
>>>> Jan 17 20:33:38.166 TRACE 1 com.cloudera.hivecommon.dataen
>>>> gine.HiveJDBCNativeQueryExecutor.HiveJDBCNativeQueryExecutor
>>>> (com.cloudera.dsi.core.impl.DSILogger@1d057a39,
>>>> com.cloudera.hivecommon.api.HS2Client@26be92ad, "select sum(1) from
>>>> dwh.event_detail where org_group_id > 10;"): +++++ enter +++++
>>>> ...
>>>>
>>>> Sunil Parmar
>>>>
>>>> On Wed, Jan 17, 2018 at 5:52 PM, Vincent Tran <vttran@cloudera.com>
>>>> wrote:
>>>>
>>>>> The properties are case sensitive so I would try UseNativeQuery just
>>>>> to dot the i's and cross the t's.
>>>>>
>>>>> Ensure that the property is recognized and applied by the driver by
>>>>> checking the JDBC log. (LogLevel=6;LogPath=/path/to/foo;)
>>>>>
>>>>> On Jan 17, 2018 6:45 PM, "Sunil Parmar" <sunilosunil@gmail.com>
wrote:
>>>>>
>>>>>> useNativeQuery didn't help.
>>>>>>
>>>>>> This is my code.
>>>>>>
>>>>>> public static void main(String[] args) throws IOException,
>>>>>> SQLException {
>>>>>>         connectionUrl =
>>>>>>      "jdbc:impala://hadoop102.qa1.sac.int.company.com:21050/dwh;
>>>>>> AuthMech=3;UID=ldaptest;PWD=test;useNativeQuery=1";
>>>>>>     jdbcDriverName = "com.cloudera.impala.jdbc41.Driver";
>>>>>>         System.out.println("\n======================================
>>>>>> =======");
>>>>>>         System.out.println("Cloudera Impala JDBC Example");
>>>>>>         System.out.println("Using Connection URL: " + connectionUrl);
>>>>>>         // System.out.println("Running Query: " + sqlStatement);
>>>>>>         Connection con = null;
>>>>>>         try {
>>>>>>             Class.forName(jdbcDriverName);
>>>>>>             con = DriverManager.getConnection(connectionUrl);
>>>>>>             String query="select count( distinct yyyymmdd) from
>>>>>> dwh.event_detail limit 1/*abcdefg1*/";
>>>>>>             System.out.println("\n== Begin Query Results
>>>>>> ======================");
>>>>>>             System.out.println("\n== Running query=== " + query);
>>>>>>             Statement st = con.createStatement();
>>>>>>             ResultSet rs = st.executeQuery(query);
>>>>>>             QueryTask.printResultSet(rs);
>>>>>>
>>>>>>         } catch (SQLException e) {
>>>>>>             e.printStackTrace();
>>>>>>         } catch (Exception e) {
>>>>>>             e.printStackTrace();
>>>>>>         } finally {
>>>>>>             con.close();
>>>>>>
>>>>>>         }
>>>>>>     }
>>>>>>
>>>>>> Same query when executed from HUE shows comments. Look at the
>>>>>> screenshot from CM.
>>>>>>
>>>>>>
>>>>>> ​
>>>>>>
>>>>>> Sunil Parmar
>>>>>>
>>>>>> On Wed, Dec 13, 2017 at 11:30 PM, Mostafa Mokhtar <
>>>>>> mmokhtar@cloudera.com> wrote:
>>>>>>
>>>>>>> +1 on what Alex recommends.
>>>>>>> Highly recommend using useNativeQuery=1 all the time.
>>>>>>>
>>>>>>> Thanks
>>>>>>> Mostafa
>>>>>>>
>>>>>>> On Dec 13, 2017, at 9:42 PM, Alexander Behm <alex.behm@cloudera.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Not sure if this will work, but you could try enabling
>>>>>>> useNativeQuery in the driver. Theoretically that should send
the query
>>>>>>> string including comments verbatim to Impala.
>>>>>>>
>>>>>>> On Wed, Dec 13, 2017 at 7:57 PM, Philip Zeyliger <
>>>>>>> philip@cloudera.com> wrote:
>>>>>>>
>>>>>>>> I’m a little surprised comments are being stripped, but
adding (1=1
>>>>>>>> or “”=“your comment”) to a where clause would work
similarly. Impala will
>>>>>>>> simplify that away during query compilation.
>>>>>>>>
>>>>>>>> P
>>>>>>>>
>>>>>>>> On Wed, Dec 13, 2017 at 7:22 PM Sunil Parmar <sunilosunil@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Server version: impalad version 2.9.0-cdh5.12.1 RELEASE
(build
>>>>>>>>> 5131a031f4aa38c1e50c430373c55ca53e0517b9)
>>>>>>>>>
>>>>>>>>> One of the primary use of of Impala is it acts as a backend
to a
>>>>>>>>> web application exposed to lot of users. We are trying
to build monitoring
>>>>>>>>> metrics per web page, logged in user and other things
that are not
>>>>>>>>> typically supplied in the query string. We were thinking
to use query
>>>>>>>>> comments to pass such information along with query. It
worked fine when
>>>>>>>>> query is executed using impala-shell; I can see the comments
in the impala
>>>>>>>>> monitoring page i.e. on port #25000 but when comments
are sent over JDBC ;
>>>>>>>>> they're not visible to Impalad.
>>>>>>>>>
>>>>>>>>> 1. Is there an configuration in Impala driver ? We're
using
>>>>>>>>>      <dependency>
>>>>>>>>>       <groupId>com.cloudera.impala</groupId>
>>>>>>>>>       <artifactId>impala-jdbc</artifactId>
>>>>>>>>>       <version>2.5.30</version>
>>>>>>>>>       <scope>compile</scope>
>>>>>>>>>     </dependency>
>>>>>>>>>
>>>>>>>>> 2. Is there any other way to send such hints along with
query ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Sunil
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>

Mime
View raw message