tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaehwa Jung (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (TAJO-433) Improve integration with Hive
Date Mon, 30 Dec 2013 18:43:50 GMT

     [ https://issues.apache.org/jira/browse/TAJO-433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jaehwa Jung updated TAJO-433:
-----------------------------

    Attachment: TAJO-433.patch

I'm glad to upload this patch. From now on, tajo can use HiveMetastore with hive-site.xml.
But verifying dependencies for hive was both tricky and time-consuming job. I updated tajo
as follows:

# HCatalogUtil: It made HiveMetaStoreClient using HiveConf. This update affected HCatalogStore
and TestHCatalogStore.
# pom.xml: some hive dependencies made trouble which load another hive libraries. So, I checked
all hive dependencies.
# tajo: Previous tajo added all jar files in hive library directory into class path. But some
jar files crashed tajo libraries. For example, tajo uses protobuf version 2.5.0, but hive
0.12. uses protobuf version 2.4.1. So, After I verified each jar file, I added it into class
path.
# tajo-env.sh: I added HIVE_JDBC_DRIVER_DIR which specify hive jdbc driver jar file path because
HCatalogStore needed it. 

You can verify this patch as follows:
{code:xml}
* for hive 0.11.0: mvn clean install -Phcatalog-0.11.0
* for hive 0.12.0: mvn clean install -Phcatalog-0.12.0
{code}


> Improve integration with Hive
> -----------------------------
>
>                 Key: TAJO-433
>                 URL: https://issues.apache.org/jira/browse/TAJO-433
>             Project: Tajo
>          Issue Type: Improvement
>          Components: catalog
>    Affects Versions: 0.8-incubating
>            Reporter: Jaehwa Jung
>            Assignee: Jaehwa Jung
>             Fix For: 0.8-incubating
>
>         Attachments: TAJO-433.patch
>
>
> Hi, guys~ 
> I wish to discuss about HCatalogStore. 
> HiveMetaStore types consists of three types: embedded, local, remote. 
> This type will set up at hive configuration file which named 'hive-site.xml' and HiveMetaStoreClient
can use MetaStore by configuration file. So, if Tajo have only to add hive configuration file
to classpath, Tajo can use HiveMetaStore with HiveMetaStoreClient. 
> But current HCatalogStore set up configurations about HiveMetaStore as follows:
> - hive.metastore.uris
> - hive.metastore.kerberos.principal
> - hive.metastore.local
> - hive.metastore.sasl.enabled
> And Tajo doesn't need 'tajo.catalog.uri' property because 'hive-site.xml' already includes
metastore host and port. So, it looks like unnecessary settings. 
> What do you think about this suggestion? :)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message