hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Huang Meilong <ims...@outlook.com>
Subject 答复: Can I specify database name in hive metastore service?
Date Thu, 27 Oct 2016 11:50:41 GMT
Because I have many clusters and I want to manage all these clusters' meta data with one metastore,
and make the clusters' meta data isolated from each other. Maybe I could create a hive JIRA.
Thank you all the same.

________________________________
发件人: Peter Vary <pvary@cloudera.com>
发送时间: 2016年10月27日 19:06:32
收件人: Huang Meilong
抄送: user@hive.apache.org
主题: Re: Can I specify database name in hive metastore service?

Hi Huang,

It is possible to use the same Metastore server for 2 HiveServer2 instances, but then the
HiveServer2 instances will share the metastore. They will have the same databases and tables.

As far as I know it is not possible for a metastore to serve different data for different
HiveServers.
Here someone with more experience might help more.

Thanks,
Peter


On Oct 27, 2016, at 4:41 AM, Huang Meilong <imsrch@outlook.com<mailto:imsrch@outlook.com>>
wrote:

Thanks Peter for your detailed explanation, it works now.

BTW, can I start 2 HiveServer2 instances with ONLY ONE remote metastore service and make the
metastore db stored in the same RDBMS in a different database?
________________________________
发件人: Peter Vary <pvary@cloudera.com<mailto:pvary@cloudera.com>>
发送时间: 2016年10月26日 21:11:26
收件人: Huang Meilong
抄送: user@hive.apache.org<mailto:user@hive.apache.org>
主题: Re: Can I specify database name in hive metastore service?

Hi Huang,

According to the picture you want to start 2 HiveServer2 instances both with embedded metastore,
where the metastore db is stored in the same RDBMS in a different database.

This is certainly possible.
You have to set the database options according to this:
https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin#AdminManualMetastoreAdmin-RemoteMetastoreDatabase

You have to set the metastore options according to this:
https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin#AdminManualMetastoreAdmin-Local/EmbeddedMetastoreServer


Hope this helps,
Peter


On Oct 26, 2016, at 4:48 AM, Huang Meilong <imsrch@outlook.com<mailto:imsrch@outlook.com>>
wrote:

Look at this local metastore architecture:

<pastedImage.png>

If I set different database name in javax.jdo.option.ConnectionURL, say,
"jdbc:mysql://xxxxx/hivemeta_1?createDatabaseIfNotExist=true&amp;characterEncoding=UTF-8"
and "jdbc:mysql://xxxxx/hivemeta_2?createDatabaseIfNotExist=true&amp;characterEncoding=UTF-8",
will the to metastore services work fine?

In short, I want to use the same RDBMS database for the two hive metastore services, and the
meta data is isolated form each other. How can I achieve that?



________________________________

发件人: Peter Vary <pvary@cloudera.com<mailto:pvary@cloudera.com>>
发送时间: 2016年10月26日 0:49
收件人: user@hive.apache.org<mailto:user@hive.apache.org>
主题: Re: Can I specify database name in hive metastore service?

Hi Huang,
Hive metastore is a component of the "Hive database". See: https://cwiki.apache.org/confluence/display/Hive/Design
The metastore uses traditional RDBMS to store "the structure information of the various tables
and partitions in the warehouse". The javax.jdo.option.ConnectionURL and the javax.jdo.option.ConnectionDriverName
configuration options are used to access this RDBMS database. The hive.metastore.uris is the
endpoint where the metastore will communicate with the other Hive components, like the HiveServer2.
So you can change the database name in the connectionUrl, which will change only the database
name where the metadata is stored in the relational database and you can not add a database
name to the thrift uri (metastore uri) since HiveServer2 will use the same uri to access metadata
regardless of which Hive database is used by the client.
I hope this helps,
Peter

2016. okt. 25. 17:32 ezt írta ("Huang Meilong" <imsrch@outlook.com<mailto:imsrch@outlook.com>>):
Hi,

To use hive metastore service, I must set `javax.jdo.option.ConnectionURL`, `javax.jdo.option.ConnectionDriverName`
and `hive.metastore.uris` in hive-site.xml, like this:


  <property>

    <name>javax.jdo.option.ConnectionURL</name>

    <value>jdbc:mysql://xxxxx/hivemeta?createDatabaseIfNotExist=true&amp;characterEncoding=UTF-8</value>

  </property>

  <property>

    <name>javax.jdo.option.ConnectionDriverName</name>

    <value>com.mysql.jdbc.Driver</value>

  </property>

<property>

    <name>hive.metastore.uris</name>

    <value>thrift://xxx:9083</value>

  </property>


I'm confused that can I change the database name (usually it's `hivemeta`) for other names?

If I changed the database name from `hivemeta` to `my_hivemeta`, can hive metastore work?
We can not specify database name in `hive.metastore.uris`, we can only specify hostname and
port of metastore service.

Mime
View raw message