hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amey Barve <ameybarv...@gmail.com>
Subject Re: Predicates for 'like' and 'between' operators to custom storage handler.
Date Thu, 05 May 2016 12:10:17 GMT
Thanks Mich,

It does work, but with operators other than '=', '!=', '<', '<=', '>' and
'>=' , my code with custom storage handler gets null expression.

*final String expression = conf.get(TableScanDesc.FILTER_EXPR_CONF_STR); //
expression is null for hive query that has like operator*

Why does above API return *null* for hive query that has like operator? I
need to pass like operator for predicate push down to my custom storage
handler.

Regards,
Amey

On Thu, May 5, 2016 at 5:30 PM, Mich Talebzadeh <mich.talebzadeh@gmail.com>
wrote:

> On a normal query using sql in* Hive 2* LIKE predicate works fine. Case
> in point in a 1 billion rows table with the column random_string of
> varchar(50) I have one row that satisfies the followinh@
>
>
> +-----------+------------------+------------------+-------------------+-----------------------------------------------------+-----------------+----------------+--+
> | dummy.id  | dummy.clustered  | dummy.scattered  | dummy.randomised
> |                 dummy.random_string                 | dummy.small_vc  |
> dummy.padding  |
>
> +-----------+------------------+------------------+-------------------+-----------------------------------------------------+-----------------+----------------+--+
> | 1         | 0                | 0                | 63                |
> rMLTDXxxqXOZnqYRJwInlGfGBTxNkAszBGEUGELqTSRnFjRGbi  |          1      |
> xxxxxxxxxx     |
> | 2         | 0                | 1                | 926               |
> UEDJsfIgoYqwreSuuvjIcPZarpxMdCthpDCsgPlJfvIiylLiBS  |          2      |
> xxxxxxxxxx     |
>
> Now let us try to select that row with LIKE predicate:
>
> 0: jdbc:hive2://rhes564:10010/default> select count(1) from dummy where
> random_string like 'rMLTDXxxqXOZnqYRJ%';
>
> INFO  :
> Query Hive on Spark job[0] stages:
> INFO  : 0
> INFO  : 1
> INFO  :
> Status: Running (Hive on Spark job[0])
>
> INFO  : Completed executing
> command(queryId=hduser_20160505125700_cbc415b6-91bb-4ed6-95e4-d177e12988f6);
> Time taken: 153.544 seconds
> INFO  : OK
> +-----+--+
> | c0  |
> +-----+--+
> | 1   |
> +-----+--+
> 1 row selected (153.959 seconds)
>
> So it does work
>
> HTH
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> On 5 May 2016 at 11:53, Amey Barve <ameybarve15@gmail.com> wrote:
>
>> Hi All,
>>
>> I have implemented custom storage-handler and able to get predicates from
>> hive for '=', '!=', '<', '<=', '>' and '>=' operators.
>> But I cannot get predicates from hive for 'like', 'between' operators.
>>
>> Here's my code:
>>
>> *final String expression = conf.get(TableScanDesc.FILTER_EXPR_CONF_STR);*
>>
>> here expression remains null for like and between operators but not null
>> for above operators.
>>
>> Does hive not give predicates for 'like' and 'between' operators to
>> custom storage handler ?
>> *Is there some other mechanism to get predicates for 'like' operator*?
>>
>> I tested with hive version 1.2 and 0.14.
>>
>> Thanks and Regards,
>> Amey
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message