impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 吴朱华 <ikew...@gmail.com>
Subject Re: About impala partitoning
Date Thu, 12 May 2016 03:38:04 GMT
thx for all feedbacks^_^

2016-05-12 0:10 GMT+08:00 John Russell <jrussell@cloudera.com>:

> Thinking of how one could do this with a BETWEEN test... how about
> synthesizing a new column, something like:
>
> create table raw (x int);
>
> insert into raw values (1),(2),(3),(4),(5),(6);
>
> create table ranges (x int) partitioned by (segment string);
>
> -- The synthesized last column in the SELECT list determines the 'segment'
> value.
> insert into ranges partition (segment) select x,
>   case
>     when x between 1 and 2 then 'small'
>     when x between 3 and 4 then 'medium'
>     when x between 5 and 6 then 'large'
>     else null
>   end
> from raw;
>
> show partitions ranges;
> +---------+-------+--------+
> | segment | #Rows | #Files |
> +---------+-------+--------+
> | large   | -1    | 1      |
> | medium  | -1    | 1      |
> | small   | -1    | 1      |
> | Total   | -1    | 3      |
> +---------+-------+--------+
>
> select * from ranges;
> +---+---------+
> | x | segment |
> +---+---------+
> | 5 | large   |
> | 6 | large   |
> | 3 | medium  |
> | 4 | medium  |
> | 1 | small   |
> | 2 | small   |
> +---+---------+
>
> John
>
> On May 10, 2016, at 10:56 PM, Alex Behm <alex.behm@cloudera.com> wrote:
>
> I may be misunderstanding. Are you asking whether a range of values can be
> mapped to a single partition? That, unfortunately, is not possible. One
> value maps to one partition. If you to multiple partition key values into a
> single partition, you'll need to manually maintain some kind of mapping
> table separate from the Impala table.
>
> On Tue, May 10, 2016 at 10:52 PM, 吴朱华 <ikewu83@gmail.com> wrote:
>
>> Can be used during partitoning creation, if the data cardinality is high?
>>
>> 2016-05-10 23:49 GMT+08:00 Alex Behm <alex.behm@cloudera.com>:
>>
>>> All predicates can be used for partition pruning, including between
>>> predicates.
>>>
>>> On Tue, May 10, 2016 at 6:20 AM, 吴朱华 <ikewu83@gmail.com> wrote:
>>>
>>>> Hi guys:
>>>>
>>>> About impala partitoning, is between filter supported? From the guides,
>>>> all I can seen is between filter.
>>>>
>>>
>>>
>>
>
>

Mime
View raw message