impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Jacobs>
Subject Re: impala, kudu and hive interaction
Date Tue, 24 Jan 2017 00:15:13 GMT
Good questions.

On Thu, Jan 19, 2017 at 1:12 AM, Рябков Алексей Николаевич
<> wrote:
> 1.       When I use DDL (create table etc) how interact all components? I
> think impalad send request to impala-catalog , then catalog update hive
> metastore, then catalog  send request to kudu-master, and kudu master send
> request to kudu-tablet. Is it correct?


> 2.       When I use DML (select, update) how interact components? I think
> impalad query metadata from impala-catalog, then impalad send DML to  other
> impalad (on node), and impalad(on node) get/set data from/to kudu-tablet. Is
> it correct?


> 3.       Why  impala use hive metastore in such case? May be impala-catalog
> just interact with kudu-master.

It's complicated, but ideally we wouldn't use HMS in addition to
Kudu's metadata -- there is duplication. We aim to improve this, but
there are some problems:
* how to have a single set of databases/tables that work across storage layers
* how to create/store metadata that isn't kept by kudu, e.g. table/col
statistics and views
* how would the authorization system work across both?
and more...


View raw message