hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elliot West <tea...@gmail.com>
Subject Retrying metastore clients
Date Mon, 16 May 2016 14:33:54 GMT
Hello,

We have a fair amount of code that uses IMetaStoreClient implementation to
talk over a network to Hive metastore instances. I was under the impression
that the recommended way to create said clients was to use:

org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HiveConf)


However, I'm keen that our clients retry in the event of adverse network
conditions and I've recently discovered that clients constructed in this
manner do not appear to be of the retrying variety. In fact it seems as
though only the following factory method creates clients of this type:

org.apache.hadoop.hive.ql.metadata.Hive.getMSC()


But, in my particular use case I wish to copy data between two metastores
and this class seems to do a bunch of ThreadLocal stuff that would appear
to make it awkward to instantiate multiple, different IMetaStoreClient
instances. Therefore I think I'll need to instead call the following
directly:

org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(HiveConf,
HiveMetaHookLoader, String)


I'd be grateful for any advice regarding how I best create multiple, robust
clients that can collaborate in a single thread.

Thanks,

Elliot.

Mime
View raw message