hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ido Hadanny <ido.hada...@gmail.com>
Subject Is a collocated join (a-la-netezza) theoretically possible in hive?
Date Fri, 05 Aug 2011 08:24:50 GMT
When you join tables which are distributed on the same key and used these
key columns in the join condition, then each SPU (machine) in netezza works
100% independent of the other (see
nz-interview<http://www.folkstalk.com/2011/06/netezza-interview-questions-part-2.html>
.)

In hive, there's bucketed map
join<https://issues.apache.org/jira/browse/HIVE-917>,
but the distribution of the files representing the tables to datanode is the
responsibility of HDFS, it's not done according to hive CLUSTERED BY key!

so suppose I have 2 tables, CLUSTERED BY the same key, and I join by that
key - can hive get a guarantee from HDFS that matching buckets will sit on
the same node? or will it always have to move the matching bucket of the
small table to the datanode containing the big table bucket?

Thanks, ido

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message