kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "huaicui (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KYLIN-3601) The max connection number generated by the PreparedContextPool is inconsistent with the configuration.
Date Mon, 15 Oct 2018 05:34:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649744#comment-16649744
] 

huaicui commented on KYLIN-3601:
--------------------------------

We use Gatling to loading test;

This is our test script:
import java.util.concurrent.TimeUnit
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._
 
class KylinSimulation2 extends Simulation {
  val httpConf = http
    .acceptHeader("application/json")
    .contentTypeHeader("application/json")
 
  val headers_json = Map(
    "Content-Type" -> "application/json",
    "Authorization" -> "Basic QURNSU46S1lMSU4="
  )
 
  val scn2 = scenario("KylinServer6").during(60) {    // 60 means keep sending requests for
about 60s
    exec(
      http("KylinQuery4")
        .post("http://sap-datanode5:7070/kylin/api/query")
        .headers(headers_json)
        .body(StringBody("{ \"sql\":\"select CLIENTAGENT, STARTDAY, COUNT(*) TOTAL_NUMBER
from XXRPT_HGSMEETINGUSERREPORT_VIEW where SITEID = ? group by STARTDAY, CLIENTAGENT\", \"project\":\"CCA_REPORT\",
\"params\":[{\"className\":\"java.lang.String\",\"value\":\"1000182\"}] }")).asJSON
    )
  }
 
  setUp(
    scn2.inject(atOnceUsers(150))    // 150 users(threads)
  ).protocols(httpConf)
}

We use 150 threads to loading test, kylin.query.statement-cache-max-num-per-key is 200; We
found the number of PrepareStatement instance would be more and more like the attaches(CreatedCount
will be larger and larger); No matter how many concurrent that will trigger this issue;

> The max connection number generated by the PreparedContextPool is inconsistent with the
configuration.
> ------------------------------------------------------------------------------------------------------
>
>                 Key: KYLIN-3601
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3601
>             Project: Kylin
>          Issue Type: Bug
>          Components: Query Engine
>    Affects Versions: v2.5.0
>            Reporter: huaicui
>            Priority: Major
>         Attachments: FirstResponseDistribute.jpg, SixthResponseDistribute.jpg, image-2018-09-28-15-14-00-288.png,
image.png
>
>
> 因为并发性能不够,使用了magang提供的PrepareStatement方法进行测试。性能有所有提高,但随着测试次数的增加,吞吐率会越来越低而且数据超时也越来越多。经过修改代码在queryAndUpdateCache最后返回前加入日志打印:logger.debug("BorrowedCount:"+preparedContextPool.getBorrowedCount()
>  +",DestroyedCount:"+preparedContextPool.getDestroyedCount()
>  +",CreatedCount:"+preparedContextPool.getCreatedCount()
>  +",ReturnedCount:"+preparedContextPool.getReturnedCount()
> 同时配置文件加入该配置:
> kylin.query.statement-cache-max-num-per-key=200
>  
>  
> 日志显示,当同一sql并发一段时间后,PreparedContextPool创建了越来越多PrepareStatement,并没有进行阻塞后续来的请求。
> !image-2018-09-28-15-14-00-288.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message