hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Chauhan <hashut...@apache.org>
Subject Re: Review Request 61009: Extend object store to store bit vectors
Date Mon, 24 Jul 2017 00:17:27 GMT


> On July 23, 2017, 4:58 p.m., Ashutosh Chauhan wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
> > Lines 1846 (patched)
> > <https://reviews.apache.org/r/61009/diff/2/?file=1780981#file1780981line1846>
> >
> >     This cast of null is not necessary.
> 
> pengcheng xiong wrote:
>     it will throw exception if i just use null for derby. It seems that derby expects
a type for null.

I see. This will throw exception for other DBs since type names are different for each one.
So, either we need to have logic for type name corresponding to each DB or have this cast
only for Derby  (I tested on mysql, sql server, postgres and oracle, there is no such requirement)
or do some other trick. For example, instead of select null, do select '' ie, empty string.


- Ashutosh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61009/#review181182
-----------------------------------------------------------


On July 22, 2017, 9:19 p.m., pengcheng xiong wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61009/
> -----------------------------------------------------------
> 
> (Updated July 22, 2017, 9:19 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-16997
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/common/ndv/FMSketch.java e20d29954a 
>   common/src/java/org/apache/hadoop/hive/common/ndv/NumDistinctValueEstimatorFactory.java
e810ac5487 
>   common/src/java/org/apache/hadoop/hive/common/ndv/fm/FMSketchUtils.java PRE-CREATION

>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLog.java d1955468a6 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java df45f2cc32 
>   common/src/test/org/apache/hadoop/hive/common/ndv/fm/TestFMSketchSerialization.java
PRE-CREATION 
>   metastore/scripts/upgrade/derby/044-HIVE-16997.derby.sql PRE-CREATION 
>   metastore/scripts/upgrade/derby/hive-schema-3.0.0.derby.sql a9a532906f 
>   metastore/scripts/upgrade/derby/upgrade-2.3.0-to-3.0.0.derby.sql 30513dc882 
>   metastore/scripts/upgrade/mssql/029-HIVE-16997.mssql.sql PRE-CREATION 
>   metastore/scripts/upgrade/mssql/hive-schema-3.0.0.mssql.sql 1cfe2d1b2d 
>   metastore/scripts/upgrade/mssql/upgrade-2.3.0-to-3.0.0.mssql.sql 5683254b04 
>   metastore/scripts/upgrade/mysql/044-HIVE-16997.mysql.sql PRE-CREATION 
>   metastore/scripts/upgrade/mysql/hive-schema-3.0.0.mysql.sql 97d881f263 
>   metastore/scripts/upgrade/mysql/upgrade-2.3.0-to-3.0.0.mysql.sql ba62939809 
>   metastore/scripts/upgrade/oracle/044-HIVE-16997.oracle.sql PRE-CREATION 
>   metastore/scripts/upgrade/oracle/hive-schema-3.0.0.oracle.sql 8fdb552367 
>   metastore/scripts/upgrade/oracle/upgrade-2.3.0-to-3.0.0.oracle.sql 0a70d47cca 
>   metastore/scripts/upgrade/postgres/043-HIVE-16997.postgres.sql PRE-CREATION 
>   metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql 1cdeb6b45a 
>   metastore/scripts/upgrade/postgres/upgrade-2.3.0-to-3.0.0.postgres.sql c44dd067fc 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java a960b2d26b

>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java b52c94c9fb

>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java db4ec91cdb 
>   metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 2dc2804343

>   metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java 3ac4fe1604

>   metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/DateColumnStatsAggregator.java
PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/StringColumnStatsAggregator.java
PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/StatsCache.java 0e119896a5

>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BinaryColumnStatsAggregator.java
d81d612e92 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BooleanColumnStatsAggregator.java
e796df2422 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregator.java
29a05390bf 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregatorFactory.java
568bf0609b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DecimalColumnStatsAggregator.java
8eb64e0143 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DoubleColumnStatsAggregator.java
b6b86123b2 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/IExtrapolatePartStatus.java
af75bced72 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/LongColumnStatsAggregator.java
2da6f60167 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/StringColumnStatsAggregator.java
83c6c54fd2 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/BinaryColumnStatsMerger.java
af0669eb65 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/BooleanColumnStatsMerger.java
33ff6a19f5 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMerger.java
d3051a2b00 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
c013ba5c5d 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
e899bfe85f 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DecimalColumnStatsMerger.java
4099ffcace 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DoubleColumnStatsMerger.java
1691fc97df 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/LongColumnStatsMerger.java
361af350fe 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/StringColumnStatsMerger.java
8e28f907ee 
>   metastore/src/model/org/apache/hadoop/hive/metastore/model/MPartitionColumnStatistics.java
2967a60fae 
>   metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableColumnStatistics.java
132f7a137b 
>   metastore/src/model/package.jdo 9c4bc219f2 
>   metastore/src/test/org/apache/hadoop/hive/metastore/TestOldSchema.java PRE-CREATION

>   metastore/src/test/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java 1fa9447145

>   metastore/src/test/org/apache/hadoop/hive/metastore/hbase/TestHBaseAggregateStatsCacheWithBitVector.java
ecc99c3300 
>   metastore/src/test/org/apache/hadoop/hive/metastore/hbase/TestHBaseAggregateStatsExtrapolation.java
99ce96ca0d 
>   metastore/src/test/org/apache/hadoop/hive/metastore/hbase/TestHBaseAggregateStatsNDVUniformDist.java
74e16695a9 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 97bf839ae1 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
aa77234c28 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ColStatistics.java 41a1c7a582 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java d7a9888389 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFComputeStats.java 2d56950cb1

>   ql/src/test/queries/clientpositive/bitvector.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/fm-sketch.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/hll.q edfdce8a29 
>   ql/src/test/results/clientpositive/alterColumnStats.q.out 519a62a190 
>   ql/src/test/results/clientpositive/alterColumnStatsPart.q.out 672bd9f4bb 
>   ql/src/test/results/clientpositive/alter_partition_update_status.q.out c0d4eeefb4 
>   ql/src/test/results/clientpositive/alter_table_column_stats.q.out 96dce1e2c5 
>   ql/src/test/results/clientpositive/alter_table_update_status.q.out 9cd9a8dbe0 
>   ql/src/test/results/clientpositive/analyze_tbl_part.q.out 6a3fbc0cc7 
>   ql/src/test/results/clientpositive/autoColumnStats_5.q.out e3abba5bd0 
>   ql/src/test/results/clientpositive/autoColumnStats_9.q.out 06f23b1e7c 
>   ql/src/test/results/clientpositive/avro_decimal.q.out e1045ebea1 
>   ql/src/test/results/clientpositive/avro_decimal_native.q.out b73b5f5679 
>   ql/src/test/results/clientpositive/bitvector.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/char_udf1.q.out fefc7407e0 
>   ql/src/test/results/clientpositive/colstats_all_nulls.q.out 0f2822504f 
>   ql/src/test/results/clientpositive/column_names_with_leading_and_trailing_spaces.q.out
fb833bccb2 
>   ql/src/test/results/clientpositive/column_pruner_multiple_children.q.out 9925928da7

>   ql/src/test/results/clientpositive/columnstats_partlvl.q.out 5ecb20501b 
>   ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out a64c76badf 
>   ql/src/test/results/clientpositive/columnstats_tbllvl.q.out 91c8f150a2 
>   ql/src/test/results/clientpositive/compustat_avro.q.out 2f8dc10e50 
>   ql/src/test/results/clientpositive/compute_stats_date.q.out 5cd2180108 
>   ql/src/test/results/clientpositive/compute_stats_decimal.q.out fcfce78b82 
>   ql/src/test/results/clientpositive/compute_stats_double.q.out e6a087dd98 
>   ql/src/test/results/clientpositive/compute_stats_long.q.out fb985d8266 
>   ql/src/test/results/clientpositive/compute_stats_string.q.out a5d66eba31 
>   ql/src/test/results/clientpositive/confirm_initial_tbl_stats.q.out 5593e422b6 
>   ql/src/test/results/clientpositive/decimal_stats.q.out f58a7cc8e1 
>   ql/src/test/results/clientpositive/deleteAnalyze.q.out 1bae859e2c 
>   ql/src/test/results/clientpositive/describe_syntax.q.out 19147a1d92 
>   ql/src/test/results/clientpositive/describe_table.q.out 3ba9a7b942 
>   ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out 73d4cd7660 
>   ql/src/test/results/clientpositive/encrypted/encryption_move_tbl.q.out 1096e9fc64 
>   ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out b212da907b 
>   ql/src/test/results/clientpositive/extrapolate_part_stats_partial.q.out b5f4feede0

>   ql/src/test/results/clientpositive/fm-sketch.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/hll.q.out b9357c3043 
>   ql/src/test/results/clientpositive/llap/autoColumnStats_2.q.out f29f7b5d1a 
>   ql/src/test/results/clientpositive/llap/column_names_with_leading_and_trailing_spaces.q.out
fb833bccb2 
>   ql/src/test/results/clientpositive/llap/columnstats_part_coltype.q.out 5e647433f1 
>   ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out 5db87d97cf 
>   ql/src/test/results/clientpositive/llap/extrapolate_part_stats_partial_ndv.q.out 6bc1970ad0

>   ql/src/test/results/clientpositive/llap/llap_smb.q.out 87b33db805 
>   ql/src/test/results/clientpositive/llap/stats_only_null.q.out 57aaf557b2 
>   ql/src/test/results/clientpositive/llap/varchar_udf1.q.out 2e9d88e343 
>   ql/src/test/results/clientpositive/llap/vector_udf1.q.out 9a164fe130 
>   ql/src/test/results/clientpositive/partial_column_stats.q.out 87d47dae22 
>   ql/src/test/results/clientpositive/partition_coltype_literals.q.out d459b36ff0 
>   ql/src/test/results/clientpositive/reduceSinkDeDuplication_pRS_key_empty.q.out 4bddd3bef8

>   ql/src/test/results/clientpositive/rename_external_partition_location.q.out 19546c38bc

>   ql/src/test/results/clientpositive/rename_table_update_column_stats.q.out 16b3a38c46

>   ql/src/test/results/clientpositive/spark/avro_decimal_native.q.out b73b5f5679 
>   ql/src/test/results/clientpositive/spark/stats_only_null.q.out 359eea3acb 
>   ql/src/test/results/clientpositive/stats_only_null.q.out 88c2114356 
>   ql/src/test/results/clientpositive/temp_table_display_colstats_tbllvl.q.out ad92058cab

>   ql/src/test/results/clientpositive/tez/explainanalyze_5.q.out 626e1fd4d0 
>   ql/src/test/results/clientpositive/tunable_ndv.q.out 437beafc0d 
> 
> 
> Diff: https://reviews.apache.org/r/61009/diff/2/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> pengcheng xiong
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message