hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thejas Nair <the...@hortonworks.com>
Subject Re: Review Request 59205: HIVE-16579: CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table
Date Sun, 14 May 2017 07:59:47 GMT


> On May 14, 2017, 7:38 a.m., Daniel Dai wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java
> > Lines 528 (patched)
> > <https://reviews.apache.org/r/59205/diff/4/?file=1717995#file1717995line554>
> >
> >     If using dirty flag, databaseCacheLock can be removed, right?

We still need some locking/synchronization to prevent things like this -


1. If (isDirtyFlag == false) {
2.  // if db gets updated in another thread while this thread is here, we lose the update.
3.    updateDBCache()
2. }

A read/write lock similar to what is used here appropriate for that. The background thread
is the one that holds the write lock.
However, not sure if we need to busy-wait with trylock though. (I figure, the intent is to
give this one higher priority). The update DbCache part should not take that long. (The busy-wait
can be dealt with later also).


- Thejas


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59205/#review174907
-----------------------------------------------------------


On May 14, 2017, 5:19 a.m., Vaibhav Gumashta wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59205/
> -----------------------------------------------------------
> 
> (Updated May 14, 2017, 5:19 a.m.)
> 
> 
> Review request for hive, Daniel Dai and Thejas Nair.
> 
> 
> Bugs: HIVE-16579
>     https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> https://issues.apache.org/jira/browse/HIVE-16579
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d6a80ae 
>   itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
91a3a38 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java b897ffa 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java b96c27e

>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 870896c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java ed19f42 
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java c1af690 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java 668499b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 5a187d8

>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java 7beee42

>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 0c7d8bb 
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
f613c30 
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
1720e37 
>   metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 0ab20d6

> 
> 
> Diff: https://reviews.apache.org/r/59205/diff/4/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message