impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Rahn <greg.r...@gmail.com>
Subject Re: Using comments in query for detailed monitoring
Date Fri, 26 Jan 2018 18:29:20 GMT
I was able to repro this as well and checking in with Simba who authors the
driver.


On Wed, Jan 17, 2018 at 9:03 PM, Sunil Parmar <sunilosunil@gmail.com> wrote:

> Tried with case change but no luck.  From the connection log.
>
> Jan 17 20:33:38.163 TRACE 1 com.cloudera.jdbc.common.SConnection.nativeSQL("select
> sum(1) from dwh.event_detail where org_group_id > 10; /*abcdefg*/"): +++++
> enter +++++
> Jan 17 20:33:38.163 TRACE 1 com.cloudera.hivecommon.core.H
> iveJDBCCommonConnection.toNativeSQL("select sum(1) from dwh.event_detail
> where org_group_id > 10; /*abcdefg*/"): +++++ enter +++++
> Jan 17 20:33:38.164 TRACE 1 com.cloudera.hivecommon.dataen
> gine.HiveJDBCDataEngine.prepare("select sum(1) from dwh.event_detail
> where org_group_id > 10; "): +++++ enter +++++
> Jan 17 20:33:38.166 TRACE 1 com.cloudera.hivecommon.dataen
> gine.HiveJDBCNativeQueryExecutor.HiveJDBCNativeQueryExecutor
> (com.cloudera.dsi.core.impl.DSILogger@1d057a39,
> com.cloudera.hivecommon.api.HS2Client@26be92ad, "select sum(1) from
> dwh.event_detail where org_group_id > 10;"): +++++ enter +++++
> ...
>
> Sunil Parmar
>
> On Wed, Jan 17, 2018 at 5:52 PM, Vincent Tran <vttran@cloudera.com> wrote:
>
>> The properties are case sensitive so I would try UseNativeQuery just to
>> dot the i's and cross the t's.
>>
>> Ensure that the property is recognized and applied by the driver by
>> checking the JDBC log. (LogLevel=6;LogPath=/path/to/foo;)
>>
>> On Jan 17, 2018 6:45 PM, "Sunil Parmar" <sunilosunil@gmail.com> wrote:
>>
>>> useNativeQuery didn't help.
>>>
>>> This is my code.
>>>
>>> public static void main(String[] args) throws IOException, SQLException {
>>>         connectionUrl =
>>>      "jdbc:impala://hadoop102.qa1.sac.int.company.com:21050/dwh;
>>> AuthMech=3;UID=ldaptest;PWD=test;useNativeQuery=1";
>>>     jdbcDriverName = "com.cloudera.impala.jdbc41.Driver";
>>>         System.out.println("\n======================================
>>> =======");
>>>         System.out.println("Cloudera Impala JDBC Example");
>>>         System.out.println("Using Connection URL: " + connectionUrl);
>>>         // System.out.println("Running Query: " + sqlStatement);
>>>         Connection con = null;
>>>         try {
>>>             Class.forName(jdbcDriverName);
>>>             con = DriverManager.getConnection(connectionUrl);
>>>             String query="select count( distinct yyyymmdd) from
>>> dwh.event_detail limit 1/*abcdefg1*/";
>>>             System.out.println("\n== Begin Query Results
>>> ======================");
>>>             System.out.println("\n== Running query=== " + query);
>>>             Statement st = con.createStatement();
>>>             ResultSet rs = st.executeQuery(query);
>>>             QueryTask.printResultSet(rs);
>>>
>>>         } catch (SQLException e) {
>>>             e.printStackTrace();
>>>         } catch (Exception e) {
>>>             e.printStackTrace();
>>>         } finally {
>>>             con.close();
>>>
>>>         }
>>>     }
>>>
>>> Same query when executed from HUE shows comments. Look at the screenshot
>>> from CM.
>>>
>>>
>>> ​
>>>
>>> Sunil Parmar
>>>
>>> On Wed, Dec 13, 2017 at 11:30 PM, Mostafa Mokhtar <mmokhtar@cloudera.com
>>> > wrote:
>>>
>>>> +1 on what Alex recommends.
>>>> Highly recommend using useNativeQuery=1 all the time.
>>>>
>>>> Thanks
>>>> Mostafa
>>>>
>>>> On Dec 13, 2017, at 9:42 PM, Alexander Behm <alex.behm@cloudera.com>
>>>> wrote:
>>>>
>>>> Not sure if this will work, but you could try enabling useNativeQuery
>>>> in the driver. Theoretically that should send the query string including
>>>> comments verbatim to Impala.
>>>>
>>>> On Wed, Dec 13, 2017 at 7:57 PM, Philip Zeyliger <philip@cloudera.com>
>>>> wrote:
>>>>
>>>>> I’m a little surprised comments are being stripped, but adding (1=1
or
>>>>> “”=“your comment”) to a where clause would work similarly. Impala
will
>>>>> simplify that away during query compilation.
>>>>>
>>>>> P
>>>>>
>>>>> On Wed, Dec 13, 2017 at 7:22 PM Sunil Parmar <sunilosunil@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Server version: impalad version 2.9.0-cdh5.12.1 RELEASE (build
>>>>>> 5131a031f4aa38c1e50c430373c55ca53e0517b9)
>>>>>>
>>>>>> One of the primary use of of Impala is it acts as a backend to a
web
>>>>>> application exposed to lot of users. We are trying to build monitoring
>>>>>> metrics per web page, logged in user and other things that are not
>>>>>> typically supplied in the query string. We were thinking to use query
>>>>>> comments to pass such information along with query. It worked fine
when
>>>>>> query is executed using impala-shell; I can see the comments in the
impala
>>>>>> monitoring page i.e. on port #25000 but when comments are sent over
JDBC ;
>>>>>> they're not visible to Impalad.
>>>>>>
>>>>>> 1. Is there an configuration in Impala driver ? We're using
>>>>>>      <dependency>
>>>>>>       <groupId>com.cloudera.impala</groupId>
>>>>>>       <artifactId>impala-jdbc</artifactId>
>>>>>>       <version>2.5.30</version>
>>>>>>       <scope>compile</scope>
>>>>>>     </dependency>
>>>>>>
>>>>>> 2. Is there any other way to send such hints along with query ?
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Sunil
>>>>>>
>>>>>>
>>>>
>>>
>

Mime
View raw message