hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Green <openkbi...@gmail.com>
Subject Re: Why hive 0.13 will initialize derby database if the metastore parameters are not set in hive-site.xml?
Date Sat, 07 Mar 2015 02:40:28 GMT
And Starting from hive 0.13, hive.stats.dbclass=fs by default.
So I am thinking autostats may not be related here.
Could be some other features need derby….

On Fri, Mar 6, 2015 at 6:37 PM, Jim Green <openkbinfo@gmail.com> wrote:

> Hi Edward,
>
> No matter if I set hive.stats.autogather=false in hive-site.xml or
> .hiverc, when I run “hive”, the derby database is still created.
>
> # hive
> hive> set hive.stats.autogather ;
> hive.stats.autogather=false
>
> # ls -altr|grep metastore_db
> drwxr-xr-x   5 root root     4096 Mar  6 18:34 metastore_db
>
> Thanks.
>
> On Fri, Mar 6, 2015 at 6:17 PM, Edward Capriolo <edlinuxguru@gmail.com>
> wrote:
>
>> Make sure hive autogather stats is false . Or aetup the stats db
>>
>>
>> On Friday, March 6, 2015, Jim Green <openkbinfo@gmail.com> wrote:
>>
>>> Hi Team,
>>>
>>> Starting from hive 0.13, if the metastore parameters are not set in
>>> hive-site.xml, but we set in .hiverc, hive will try to initialize derby
>>> database in current working directory.
>>> This behavior did not exist in hive 0.12.
>>> Is it a known bug? or behavior change?
>>>
>>> I have the repro as below:
>>>
>>> *Env:*
>>> Hive 0.13
>>> MySQL as backend metastore database.
>>> No hive metastore service.
>>>
>>> *Case 1:*
>>> .hiverc is not used and hive-site.xml has below 4 parameters:
>>> <property>
>>>   <name>javax.jdo.option.ConnectionURL</name>
>>>   <value>jdbc:mysql://localhost/metastore</value>
>>>   <description>the URL of the MySQL database</description>
>>> </property>
>>>
>>> <property>
>>>   <name>javax.jdo.option.ConnectionDriverName</name>
>>>   <value>com.mysql.jdbc.Driver</value>
>>> </property>
>>>
>>> <property>
>>>   <name>javax.jdo.option.ConnectionUserName</name>
>>>   <value>hive</value>
>>> </property>
>>>
>>> <property>
>>>   <name>javax.jdo.option.ConnectionPassword</name>
>>>   <value>mypassword</value>
>>> </property>
>>>
>>> In this case, if we run "hive" and it works fine and connects to mysql
>>> as the backend metastore database.
>>> It will NOT initialize the derby database in current directory.
>>>
>>> *Case 2:*
>>> hive-site.xml is empty and .hiverc has below 3 parameters:
>>> [root@~]# cat .hiverc
>>> set javax.jdo.option.ConnectionURL=jdbc:mysql://localhost/metastore;
>>> set javax.jdo.option.ConnectionDriverName=com.mysql.jdbc.Driver;
>>> set javax.jdo.option.ConnectionUserName=hive;
>>> set javax.jdo.option.ConnectionPassword=mypassword;
>>>
>>> In this case, if we run "hive" and it also works fine and connects to
>>> mysql as the backend metastore database.
>>> However it initialized the derby database in current working directory
>>> where you run "hive" command:
>>>
>>> drwxr-xr-x   5 root root   4096 Mar  6 12:18 metastore_db
>>> -rw-r--r--   1 root root  70754 Mar  6 12:18 derby.log
>>>
>>> If we open another putty session and run "hive" in the same directory,
>>> it will fail with below error:
>>> "Caused by: ERROR XSDB6: Another instance of Derby may have already
>>> booted the database /xxx/xxx/xxx/metastore_db."
>>> This is because derby database only allows one connection.
>>>
>>> We do not understand why after we moved the 4 parameters from
>>> hive-site.xml to .hiverc, hive will try to initialize the derby database?
>>>
>>>
>>> --
>>> Thanks,
>>> www.openkb.info
>>> (Open KnowledgeBase for Hadoop/Database/OS/Network/Tool)
>>>
>>
>>
>> --
>> Sorry this was sent from mobile. Will do less grammar and spell check
>> than usual.
>>
>
>
>
> --
> Thanks,
> www.openkb.info
> (Open KnowledgeBase for Hadoop/Database/OS/Network/Tool)
>



-- 
Thanks,
www.openkb.info
(Open KnowledgeBase for Hadoop/Database/OS/Network/Tool)

Mime
View raw message