kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "huaicui (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (KYLIN-3601) The max connection number generated by the PreparedContextPool is inconsistent with the configuration.
Date Thu, 18 Oct 2018 07:58:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654778#comment-16654778
] 

huaicui edited comment on KYLIN-3601 at 10/18/18 7:57 AM:
----------------------------------------------------------

Yes, I know. We use a backend service to call kylin service as a database. But a page of the
front end will send 20 query requirements, and a kylin service just support less than 50 concurrency
will be a bottleneck. If there are 100 persons open this page at the same time, will generate
20*100=2000 query requirements, we need more than 40 kylin server to support this scenario.
How do we for this issue?


was (Author: kofiori):
Yes, I know. We use a backend service to call kylin service as a database. But a page of the
front end will send 20 query requirements, and a kylin service just support less than 50 concurrency
will be a bottleneck. If there are 100 persons open this page at the same time, will generate
20*100=2000 query requirements, we need more than 40 kylin server to support this scenario.

> The max connection number generated by the PreparedContextPool is inconsistent with the
configuration.
> ------------------------------------------------------------------------------------------------------
>
>                 Key: KYLIN-3601
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3601
>             Project: Kylin
>          Issue Type: Bug
>          Components: Query Engine
>    Affects Versions: v2.5.0
>            Reporter: huaicui
>            Priority: Major
>         Attachments: FirstResponseDistribute.jpg, SixthResponseDistribute.jpg, image-2018-09-28-15-14-00-288.png,
image.png
>
>
> 因为并发性能不够,使用了magang提供的PrepareStatement方法进行测试。性能有所有提高,但随着测试次数的增加,吞吐率会越来越低而且数据超时也越来越多。经过修改代码在queryAndUpdateCache最后返回前加入日志打印:logger.debug("BorrowedCount:"+preparedContextPool.getBorrowedCount()
>  +",DestroyedCount:"+preparedContextPool.getDestroyedCount()
>  +",CreatedCount:"+preparedContextPool.getCreatedCount()
>  +",ReturnedCount:"+preparedContextPool.getReturnedCount()
> 同时配置文件加入该配置:
> kylin.query.statement-cache-max-num-per-key=200
>  
>  
> 日志显示,当同一sql并发一段时间后,PreparedContextPool创建了越来越多PrepareStatement,并没有进行阻塞后续来的请求。
> !image-2018-09-28-15-14-00-288.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message