hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin" <ser...@hortonworks.com>
Subject Re: Review Request 40467: HIVE-12075 analyze for file metadata
Date Thu, 26 Nov 2015 01:50:42 GMT


> On Nov. 19, 2015, 11:51 p.m., Alan Gates wrote:
> > metastore/if/hive_metastore.thrift, line 781
> > <https://reviews.apache.org/r/40467/diff/1/?file=1131717#file1131717line781>
> >
> >     You allow it to request caching for one partition or all.  Are there cases where
you'd want to cache some put not all partitions?  Should partName be list instead?

There is no syntax in Hive to support this now (analyze takes either a table or one partition),
we can add it later if needed.


> On Nov. 19, 2015, 11:51 p.m., Alan Gates wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataManager.java, line
111
> > <https://reviews.apache.org/r/40467/diff/1/?file=1131719#file1131719line111>
> >
> >     Is this going to work with ACID?  There's an extra level of directories there
for base and delta.

yes, the HDFS call is recursive


> On Nov. 19, 2015, 11:51 p.m., Alan Gates wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/PartitionExpressionProxy.java,
line 73
> > <https://reviews.apache.org/r/40467/diff/1/?file=1131725#file1131725line73>
> >
> >     Why did you make these methods ORC specific?  That doesn't seem appropriate
this level.  What's to keep Parquet or another format from supporting file metadata?

if they want to, they'd need to implement their own methods. The contents of the buffer are
format-specific because they are stored by a format handler.


- Sergey


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40467/#review107267
-----------------------------------------------------------


On Nov. 19, 2015, 2:37 a.m., Sergey Shelukhin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40467/
> -----------------------------------------------------------
> 
> (Updated Nov. 19, 2015, 2:37 a.m.)
> 
> 
> Review request for hive, Alan Gates and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> see jira
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2185f85 
>   itests/src/test/resources/testconfiguration.properties a33e720 
>   llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java
1d0fdf0 
>   metastore/if/hive_metastore.thrift bb754f1 
>   metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java 7c3525a

>   metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataManager.java PRE-CREATION

>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java a835f6a 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c5e7a5f

>   metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java aa96f77 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 02cbd76 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 803c6e7 
>   metastore/src/java/org/apache/hadoop/hive/metastore/PartitionExpressionProxy.java ed59829

>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 5b36b03 
>   metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java
14189da 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseReadWrite.java 2fb3e8f

>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 98e6c75 
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
9a1d159 
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
8dde0af 
>   metastore/src/test/org/apache/hadoop/hive/metastore/MockPartitionExpressionForMetastore.java
d72bf76 
>   metastore/src/test/org/apache/hadoop/hive/metastore/TestObjectStore.java 9089d1c 
>   metastore/src/test/org/apache/hadoop/hive/metastore/hbase/MockUtils.java 983129a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 9ab3e98 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 488d923 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionExpressionForMetastore.java
f9978b4 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/AnalyzeCommandUtils.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsSemanticAnalyzer.java 543bc0f

>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java eea2fcc 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g e9fbfb1 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 009e72e 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 0affe84 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CacheMetadataDesc.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java a4c3db1 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java af7e43e 
>   ql/src/test/queries/clientpositive/stats_filemetadata.q PRE-CREATION 
>   ql/src/test/results/clientpositive/tez/stats_filemetadata.q.out PRE-CREATION 
>   shims/common/src/main/java/org/apache/hadoop/hive/io/HdfsUtils.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/40467/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message