impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sunil Parmar <sunilosu...@gmail.com>
Subject Re: Using comments in query for detailed monitoring
Date Sat, 17 Feb 2018 00:52:25 GMT
Thanks is there a JIRA we can track ?

Sunil Parmar

On Tue, Feb 13, 2018 at 3:18 PM, Greg Rahn <greg.rahn@gmail.com> wrote:

> An upcoming release of the Impala driver will include a fix for not
> removing comments when UseNativeQuery=1 is used.
>
> On Fri, Jan 26, 2018 at 10:29 AM, Greg Rahn <greg.rahn@gmail.com> wrote:
>
>> I was able to repro this as well and checking in with Simba who authors
>> the driver.
>>
>>
>> On Wed, Jan 17, 2018 at 9:03 PM, Sunil Parmar <sunilosunil@gmail.com>
>> wrote:
>>
>>> Tried with case change but no luck.  From the connection log.
>>>
>>> Jan 17 20:33:38.163 TRACE 1 com.cloudera.jdbc.common.SConnection.nativeSQL("select
>>> sum(1) from dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +++++
>>> enter +++++
>>> Jan 17 20:33:38.163 TRACE 1 com.cloudera.hivecommon.core.H
>>> iveJDBCCommonConnection.toNativeSQL("select sum(1) from
>>> dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +++++ enter +++++
>>> Jan 17 20:33:38.164 TRACE 1 com.cloudera.hivecommon.dataen
>>> gine.HiveJDBCDataEngine.prepare("select sum(1) from dwh.event_detail
>>> where org_group_id > 10; "): +++++ enter +++++
>>> Jan 17 20:33:38.166 TRACE 1 com.cloudera.hivecommon.dataen
>>> gine.HiveJDBCNativeQueryExecutor.HiveJDBCNativeQueryExecutor
>>> (com.cloudera.dsi.core.impl.DSILogger@1d057a39,
>>> com.cloudera.hivecommon.api.HS2Client@26be92ad, "select sum(1) from
>>> dwh.event_detail where org_group_id > 10;"): +++++ enter +++++
>>> ...
>>>
>>> Sunil Parmar
>>>
>>> On Wed, Jan 17, 2018 at 5:52 PM, Vincent Tran <vttran@cloudera.com>
>>> wrote:
>>>
>>>> The properties are case sensitive so I would try UseNativeQuery just to
>>>> dot the i's and cross the t's.
>>>>
>>>> Ensure that the property is recognized and applied by the driver by
>>>> checking the JDBC log. (LogLevel=6;LogPath=/path/to/foo;)
>>>>
>>>> On Jan 17, 2018 6:45 PM, "Sunil Parmar" <sunilosunil@gmail.com> wrote:
>>>>
>>>>> useNativeQuery didn't help.
>>>>>
>>>>> This is my code.
>>>>>
>>>>> public static void main(String[] args) throws IOException,
>>>>> SQLException {
>>>>>         connectionUrl =
>>>>>      "jdbc:impala://hadoop102.qa1.sac.int.company.com:21050/dwh;
>>>>> AuthMech=3;UID=ldaptest;PWD=test;useNativeQuery=1";
>>>>>     jdbcDriverName = "com.cloudera.impala.jdbc41.Driver";
>>>>>         System.out.println("\n======================================
>>>>> =======");
>>>>>         System.out.println("Cloudera Impala JDBC Example");
>>>>>         System.out.println("Using Connection URL: " + connectionUrl);
>>>>>         // System.out.println("Running Query: " + sqlStatement);
>>>>>         Connection con = null;
>>>>>         try {
>>>>>             Class.forName(jdbcDriverName);
>>>>>             con = DriverManager.getConnection(connectionUrl);
>>>>>             String query="select count( distinct yyyymmdd) from
>>>>> dwh.event_detail limit 1/*abcdefg1*/";
>>>>>             System.out.println("\n== Begin Query Results
>>>>> ======================");
>>>>>             System.out.println("\n== Running query=== " + query);
>>>>>             Statement st = con.createStatement();
>>>>>             ResultSet rs = st.executeQuery(query);
>>>>>             QueryTask.printResultSet(rs);
>>>>>
>>>>>         } catch (SQLException e) {
>>>>>             e.printStackTrace();
>>>>>         } catch (Exception e) {
>>>>>             e.printStackTrace();
>>>>>         } finally {
>>>>>             con.close();
>>>>>
>>>>>         }
>>>>>     }
>>>>>
>>>>> Same query when executed from HUE shows comments. Look at the
>>>>> screenshot from CM.
>>>>>
>>>>>
>>>>> ​
>>>>>
>>>>> Sunil Parmar
>>>>>
>>>>> On Wed, Dec 13, 2017 at 11:30 PM, Mostafa Mokhtar <
>>>>> mmokhtar@cloudera.com> wrote:
>>>>>
>>>>>> +1 on what Alex recommends.
>>>>>> Highly recommend using useNativeQuery=1 all the time.
>>>>>>
>>>>>> Thanks
>>>>>> Mostafa
>>>>>>
>>>>>> On Dec 13, 2017, at 9:42 PM, Alexander Behm <alex.behm@cloudera.com>
>>>>>> wrote:
>>>>>>
>>>>>> Not sure if this will work, but you could try enabling useNativeQuery
>>>>>> in the driver. Theoretically that should send the query string including
>>>>>> comments verbatim to Impala.
>>>>>>
>>>>>> On Wed, Dec 13, 2017 at 7:57 PM, Philip Zeyliger <philip@cloudera.com
>>>>>> > wrote:
>>>>>>
>>>>>>> I’m a little surprised comments are being stripped, but adding
(1=1
>>>>>>> or “”=“your comment”) to a where clause would work similarly.
Impala will
>>>>>>> simplify that away during query compilation.
>>>>>>>
>>>>>>> P
>>>>>>>
>>>>>>> On Wed, Dec 13, 2017 at 7:22 PM Sunil Parmar <sunilosunil@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Server version: impalad version 2.9.0-cdh5.12.1 RELEASE (build
>>>>>>>> 5131a031f4aa38c1e50c430373c55ca53e0517b9)
>>>>>>>>
>>>>>>>> One of the primary use of of Impala is it acts as a backend
to a
>>>>>>>> web application exposed to lot of users. We are trying to
build monitoring
>>>>>>>> metrics per web page, logged in user and other things that
are not
>>>>>>>> typically supplied in the query string. We were thinking
to use query
>>>>>>>> comments to pass such information along with query. It worked
fine when
>>>>>>>> query is executed using impala-shell; I can see the comments
in the impala
>>>>>>>> monitoring page i.e. on port #25000 but when comments are
sent over JDBC ;
>>>>>>>> they're not visible to Impalad.
>>>>>>>>
>>>>>>>> 1. Is there an configuration in Impala driver ? We're using
>>>>>>>>      <dependency>
>>>>>>>>       <groupId>com.cloudera.impala</groupId>
>>>>>>>>       <artifactId>impala-jdbc</artifactId>
>>>>>>>>       <version>2.5.30</version>
>>>>>>>>       <scope>compile</scope>
>>>>>>>>     </dependency>
>>>>>>>>
>>>>>>>> 2. Is there any other way to send such hints along with query
?
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Sunil
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>
>>
>

Mime
View raw message