hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Barna Zsombor Klara <zsombor.kl...@cloudera.com>
Subject Re: Review Request 57503: HIVE-16024: MSCK Repair Requires nonstrict hive.mapred.mode
Date Thu, 16 Mar 2017 11:38:48 GMT


> On March 15, 2017, 4:05 a.m., Naveen Gangam wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
> > Line 781 (original), 781 (patched)
> > <https://reviews.apache.org/r/57503/diff/1/?file=1661200#file1661200line781>
> >
> >     We already a few config properties around the max batch sizes for tables/partitions.
Would it make sense to re-use one of them? They are for the same purpose, whether it be for
read or write.
> >     
> >     METASTORE_BATCH_RETRIEVE_TABLE_PARTITION_MAX
> >     METASTORE_DIRECT_SQL_PARTITION_BATCH_SIZE
> >     HIVE_MSCK_REPAIR_BATCH_SIZE
> >     METASTORE_BATCH_RETRIEVE_MAX

Those settings are a bit too low for this purpose. Here we want to limit the number of check
results (lists of a pair of 2 strings) so the value should be a lot higher than the max number
of partition objects.


- Barna Zsombor


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57503/#review168969
-----------------------------------------------------------


On March 10, 2017, 10:36 a.m., Barna Zsombor Klara wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57503/
> -----------------------------------------------------------
> 
> (Updated March 10, 2017, 10:36 a.m.)
> 
> 
> Review request for hive, Peter Vary, Sergio Pena, Sahil Takiar, and Vihang Karajgaonkar.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-16024: MSCK Repair Requires nonstrict hive.mapred.mode
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/common/FixedSizeCollection.java PRE-CREATION

>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a479deb7c0c6b779277f1029009b7dfab6dcb9e3

>   common/src/test/org/apache/hadoop/hive/common/TestFixedSizeCollection.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMetaStoreChecker.java 6805c17a116f5ef0febd36c59d454fa631ae0024

>   ql/src/test/queries/clientnegative/msck_repair_4.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/msck_repair_0.q ce8ef426a2a58845afc8333259d66725db416584

>   ql/src/test/results/clientnegative/msck_repair_4.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/msck_repair_0.q.out 3f2fe75b194f1248bd5c073dd7db6b71b2ffc2ba

> 
> 
> Diff: https://reviews.apache.org/r/57503/diff/1/
> 
> 
> Testing
> -------
> 
> Tested locally and added qtests/unit tests.
> 
> 
> Thanks,
> 
> Barna Zsombor Klara
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message