kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Yang <liy...@apache.org>
Subject Re: configuration for HBase region count
Date Tue, 05 Jan 2016 09:56:18 GMT
Or if the exact version of Kylin is provided, we can looked into the code
and deduce the wrong.

On Tue, Jan 5, 2016 at 10:32 AM, ShaoFeng Shi <shaofengshi@apache.org>
wrote:

> Zhong,
> I guess you're using Kylin v1.1; For the two "kylin.hbase.region.count.min"
> in kylin.properties, that was a bug and was fixed in v1.2 release, please
> check https://issues.apache.org/jira/browse/KYLIN-1098 Please manually
> change the second to "kylin.hbase.region.count.max", or upgrade to latest
> v1.2;
>
> Regarding the region count issue, it is based on estimation and the
> accuracy would be impacted a little by compaction; but from 500 to 900 this
> is abnormal;  we need some logs to analysis, if you can find the reducer
> stdout log of "Calculate HTable Region Splits" step, that would be helpful;
> The log looks like:
>
> Log Type: stdout
>
> Log Length: 1067
>
> 3.406: [GC [PSYoungGen: 290816K->15832K(338944K)] 290816K->15912K(1112064K), 0.0331610
secs] [Times: user=0.19 sys=0.01, real=0.03 secs]
> 1 regions
>
> 1 GB per region
>
> Thanks;
>
>
> 2016-01-05 1:24 GMT+08:00 Zhang, Zhong <zzhang@cardlytics.com>:
>
>> Hi All,
>>
>>
>>
>> Happy new year!
>>
>>
>>
>> The following rows are copied from the default Kylin
>>
>> configuration file:
>>
>> # HBase min and max region count
>>
>> kylin.hbase.region.count.min=1
>>
>> kylin.hbase.region.count.min=500
>>
>> I guess the 500 should be max?
>>
>>
>>
>> When I built my first cube, I did not change anything
>>
>> in the configuration file which means we set the
>>
>> minimum region count twice. And the following is
>>
>> one of the HTable I got:
>>
>> *HTable:* KYLIN_O03ZWB4DK9
>>
>>    - Region Count: 979
>>    - Size: 5.75 TB
>>    - Start Time: 2011-12-31 00:00:00
>>    - End Time: 2014-05-01 01:00:00
>>
>> The region count is 979 which is definitely larger than
>>
>> 500. Does it mean that the setting actually did not work?
>>
>>
>>
>> Best regards,
>>
>> Zhong
>>
>>
>>
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>
>

Mime
View raw message