hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Na Yang (JIRA)" <>
Subject [jira] [Commented] (HIVE-9119) ZooKeeperHiveLockManager does not use zookeeper in the proper way
Date Thu, 25 Dec 2014 23:25:13 GMT


Na Yang commented on HIVE-9119:

[~leftylev], thank you very much for reviewing the patch. I will take your suggestions. Regarding
to your questions, please see my answer below:

2.What are the units for hive.zookeeper.connection.basesleeptime? A TimeValidator could be
used here – see comment on HIVE-6679 for an example.

- [Na]: The unit is millisecond. I will follow the example when I upload a new patch.

3. Is the omission of an "E" for ZOOKEEPR deliberate in HIVE_ZOOKEEPR_CONNECTION_BASESLEEPTIME?
It occurs once later in the code, also without the E.

-[Na]: It is a typo. I wil correct it in the new patch.

4. Just curious: What's initial about the basesleeptime?

-[Na]: CuratorFramework uses ExponentialBackoffRetryPolicy to reconnect to the ZooKeeper server.
This retry policy retries a set number of times with increasing sleep time between retries.
The basesleeptime is the sleep time for the first retry. I will explain it more clearly in
the new patch.

Currently, the qtests do not run properly with the CuratorFramework change. So I need to work
on that and upload a new patch with these doc changes later on.  

> ZooKeeperHiveLockManager does not use zookeeper in the proper way
> -----------------------------------------------------------------
>                 Key: HIVE-9119
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Locking
>    Affects Versions: 0.13.0, 0.14.0, 0.13.1
>            Reporter: Na Yang
>            Assignee: Na Yang
>         Attachments: HIVE-9119.1.patch
> ZooKeeperHiveLockManager does not use zookeeper in the proper way. 
> Currently a new zookeeper client instance is created for each getlock/releaselock query
which sometimes causes the number of open connections between
> HiveServer2 and ZooKeeper exceed the max connection number that zookeeper server allows.

> To use zookeeper as a distributed lock, there is no need to create a new zookeeper instance
for every getlock try. A single zookeeper instance could be reused and shared by ZooKeeperHiveLockManagers.

This message was sent by Atlassian JIRA

View raw message