kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "fengYu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KYLIN-1826) kylin support more than one hive based on different hadoop claster
Date Mon, 27 Jun 2016 07:59:52 GMT
fengYu created KYLIN-1826:
-----------------------------

             Summary: kylin support more than one hive based on different hadoop claster
                 Key: KYLIN-1826
                 URL: https://issues.apache.org/jira/browse/KYLIN-1826
             Project: Kylin
          Issue Type: Improvement
          Components: Environment 
    Affects Versions: v1.5.2
            Reporter: fengYu
            Assignee: fengYu


Currently, kylin only support one hive which should run by 'hive' command, However, when source
data located in more than one hive we should deploy more kylin instance and more than one
metastore. which is difficult to manager and may cause some conflict.

I has been working on it Recently, In our cluster, there are some hive client(different metastore)
which based on different hadoop cluster, I add a new hive source type which called 'external
hive' in kylin 1.5.x

Thanks to kylin Plug-in architecture in 2.x, which make this work easiler. the main modification
are:
1. add hive root directory in hive config file, external hive client exist in this directory.
hive named by directory name.
2. add hive-site.xml file while loading hive tables.
3. store hive name into project, one project can only take one hive as source.
4. change and add some job to support job building.

I will upload my patch if I finish all my tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message