impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Russell <jruss...@cloudera.com>
Subject Re: About impala partitoning
Date Wed, 11 May 2016 16:10:57 GMT
Thinking of how one could do this with a BETWEEN test... how about synthesizing a new column,
something like:

create table raw (x int);

insert into raw values (1),(2),(3),(4),(5),(6);

create table ranges (x int) partitioned by (segment string);

-- The synthesized last column in the SELECT list determines the 'segment' value.
insert into ranges partition (segment) select x,
  case
    when x between 1 and 2 then 'small'
    when x between 3 and 4 then 'medium'
    when x between 5 and 6 then 'large'
    else null
  end
from raw;

show partitions ranges;
+---------+-------+--------+
| segment | #Rows | #Files |
+---------+-------+--------+
| large   | -1    | 1      |
| medium  | -1    | 1      |
| small   | -1    | 1      |
| Total   | -1    | 3      |
+---------+-------+--------+

select * from ranges;
+---+---------+
| x | segment |
+---+---------+
| 5 | large   |
| 6 | large   |
| 3 | medium  |
| 4 | medium  |
| 1 | small   |
| 2 | small   |
+---+---------+

John

> On May 10, 2016, at 10:56 PM, Alex Behm <alex.behm@cloudera.com> wrote:
> 
> I may be misunderstanding. Are you asking whether a range of values can be mapped to
a single partition? That, unfortunately, is not possible. One value maps to one partition.
If you to multiple partition key values into a single partition, you'll need to manually maintain
some kind of mapping table separate from the Impala table.
> 
> On Tue, May 10, 2016 at 10:52 PM, 吴朱华 <ikewu83@gmail.com <mailto:ikewu83@gmail.com>>
wrote:
> Can be used during partitoning creation, if the data cardinality is high?
> 
> 
> 2016-05-10 23:49 GMT+08:00 Alex Behm <alex.behm@cloudera.com <mailto:alex.behm@cloudera.com>>:
> All predicates can be used for partition pruning, including between predicates.
> 
> On Tue, May 10, 2016 at 6:20 AM, 吴朱华 <ikewu83@gmail.com <mailto:ikewu83@gmail.com>>
wrote:
> Hi guys:
> 
> About impala partitoning, is between filter supported? From the guides, all I can seen
is between filter.
> 
> 
> 


Mime
View raw message