hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <jornfra...@gmail.com>
Subject Re: Dynamic partition pruning
Date Wed, 19 Dec 2018 08:28:09 GMT
Can you please provide us more details:
Number of rows in each table and per partition, the table structure, hive version, table format,
is table sorted or partitioned on dt?

Why don’t you use a join, potentially with a mapjoin hint?

> Am 19.12.2018 um 09:02 schrieb Prabhakar Reddy <prabha.cloud@gmail.com>:
> 
> Hello,
> 
> I have a table large_table with more than 50K partitions and when I run below query it
is running for ever.The other table small_table2 has only five partitions and when ever I
run below query it seems to be scanning all partitions rather than scanning only five partitions
which are there in smaller table.
> 
> select * from large_table a  where a.dt in (select dt from small_table2) limit 5;
> 
> Could you please confirm if this is the expected behavior or any way we can tune this
query to fetch results faster?
> 
> Regards
> Prabhakar Reddy

Mime
View raw message