kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Billy Liu <billy...@apache.org>
Subject Re: Problem with query cache
Date Wed, 11 Jan 2017 02:42:27 GMT
What kind of query you are using? How much the result set would be? How
many HBase nodes you are using?

2017-01-11 10:11 GMT+08:00 Phong Pham <phongpham1805@gmail.com>:

> Hi,
>    i try to edit config on ehcache_test.xml like this:
>   <ehcache maxBytesLocalHeap="2048M">
>
> And i still not use LDAP authentication.   And its still not working.
> Result is:
> *Duration: 3.735*
> *Project: metrixa_global_database*
> *Realization Names: [xxx]*
> *Cuboid Ids: [45971]*
> *Total scan count: 556952*
> *Result row count: 50000*
> *Accept Partial: true*
> *Is Partial Result: false*
> *Hit Exception Cache: false*
> *Storage cache used: false*
> *Message: null*
>
> Maybe i will try LDAP and report result for you.
>
> And one more problem that i need your help is: *  "Query performance is
> not stable"*
> My query offent take 2-3s. However after a long time i execute that query,
> it takes about 10s. On log file, i saw that process will load snapshot data
> again. I try to configure *kylin.table.snapshot.max_mb=1000.*
> So is this ok?
>
> Thanks for your help.
>
> 2017-01-10 16:53 GMT+07:00 ShaoFeng Shi <shaofengshi@apache.org>:
>
>> Hi Phong,
>>
>> Did you enable "LDAP" authentication in Kylin? By default Kylin
>> configures to use 10MB memory cache only; if LDAP is enabled (it thought be
>> used in production), will use 2GB memory cache.
>>
>> You can try to allocate more memory by:
>>
>> cd $KYLIN_HOME
>> vi ./tomcat/webapps/kylin/WEB-INF/classes/ehcache_test.xml
>> In there modify "maxBytesLocalHeap="10M"" to a bigger value like 1G
>>
>> Then restart Kylin.
>>
>> I'm not sure whether it can solve the error, but it worth a try. Please
>> share the result after apply this. Thanks!
>>
>>
>> 2017-01-10 14:39 GMT+08:00 Phong Pham <phongpham1805@gmail.com>:
>>
>>> Hi,
>>>    I'm still facing with cache problems. After setting:
>>>
>>>
>>>
>>>
>>> *kylin.query.cache-threshold-duration=2kylin.query.cache-threshold-scan-count=5000kylin.query.cache-enabled=true*
>>>
>>> Result is still:
>>> *Is Partial Result: false*
>>> *Hit Exception Cache: false*
>>> *Storage cache used: false*
>>>
>>> i saw some warnings that:
>>> The configured limit of 1,000 object references was reached while
>>> attempting to calculate the size of the object graph. Severe performance
>>> degradation could occur if the sizing operation continues. This can be
>>> avoided by setting the CacheManger or Cache <sizeOfPolicy> elements
>>> maxDepthExceededBehavior to "abort" or adding stop points with
>>> @IgnoreSizeOf annotations. If performance degradation is NOT an issue at
>>> the configured limit, raise the limit value using the CacheManager or Cache
>>> <sizeOfPolicy> elements maxDepth attribute. For more information, see the
>>> Ehcache configuration documentation.
>>> 2017-01-10 00:46:48,423 INFO  [Query 25496a78-149a-4752-8c3b-d988a786b0ec-312]
>>> service.QueryService:284 :
>>>
>>> So how can i fix this?
>>> thanks
>>>
>>> 2017-01-10 9:27 GMT+07:00 ShaoFeng Shi <shaofengshi@apache.org>:
>>>
>>>> Hi Phong,
>>>>
>>>> Kylin cache query results in two scenarios:
>>>>
>>>> 1) the query time exceeds 2 seconds (defined by
>>>> kylin.query.cache-threshold-duration=2000)
>>>> or 2) the query scan lager than 10 thousands rows in hbase (defined by
>>>> kylin.query.cache-threshold-scan-count=10240)
>>>>
>>>> This means, the very fast and small query will not be hit by cache. You
>>>> can give smaller values to the above parameters to use cache more
>>>> frequently.
>>>>
>>>> 2017-01-10 9:03 GMT+08:00 Phong Pham <phongpham1805@gmail.com>:
>>>>
>>>>> How can i turn on cache for queries?
>>>>>
>>>>> I turn on *kylin.query.cache.enabled=true and enabled on System tab.
*
>>>>> However when i query and submit query again. Result still is:
>>>>>
>>>>> Accept Partial: true
>>>>> Is Partial Result: false
>>>>> Hit Exception Cache: false
>>>>> Storage cache used: false
>>>>>
>>>>> Kylin Version: 1.6.2
>>>>> Thanks for your help.
>>>>>
>>>>> 2017-01-09 20:36 GMT+07:00 Billy Liu <billyliu@apache.org>:
>>>>>
>>>>>> *Hit Exception Cache: false *means your query does not hit not
>>>>>> cache,  if you query again, the result would be "Hit Exception Cache:
true"
>>>>>>
>>>>>> 2017-01-09 17:20 GMT+08:00 Phong Pham <phongpham1805@gmail.com>:
>>>>>>
>>>>>>> Hi,
>>>>>>>     i saw this on log file after a query finished:
>>>>>>>
>>>>>>> *Accept Partial: true*
>>>>>>> *Is Partial Result: false*
>>>>>>> *Hit Exception Cache: false*
>>>>>>> *Storage cache used: false*
>>>>>>> *Message: null*
>>>>>>>
>>>>>>> On setting, i set:
>>>>>>> *kylin.query.cache.enabled=true*
>>>>>>>
>>>>>>> and enable cache on System tab.
>>>>>>>
>>>>>>> So Is my query not cached?
>>>>>>>
>>>>>>> And sometime, my kylin server go down with some bad query or
unknow
>>>>>>> reasons. So how can i kill slow queries automatically?
>>>>>>> Thanks
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>>
>>>> Shaofeng Shi 史少锋
>>>>
>>>>
>>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>>
>>
>

Mime
View raw message