kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: The coprocessor thread stopped itself due to scan timeout or scan threshold(check region server log)
Date Wed, 02 Nov 2016 09:30:54 GMT
Kylin will proctect the coprocessor from taking too much memory on HBase,
so it will estimate a threshold for each query; but the estimation might be
a little pessimistic (especially for 'count distinct');  You can set the
following parameters in conf/kylin.properties to a bigger value (this is
the default setting):

*kylin.query.coprocessor.mem.gb <http://kylin.query.coprocessor.mem.gb>=3.0*

*kylin.query.mem.budget=*3221225472

Besides, you may need check the log in region server, and considering to
optimize the query to reduce the scan numbers; In future release Kylin will
optimize this logic.

2016-11-02 14:52 GMT+08:00 沙漠火狐 <278211813@qq.com>:

> Hi
>    my Kylin version is 1.5.4.1 for CDH5.7/5.8
>
>
>   when I query in kylin, it return a error  like
>
>
> {cellset: null, rowTotalsLists: null, colTotalsLists: null, runtime:
> null,…}cellset:nullcolTotalsLists:nullerror:"IOException:
> org.apache.commons.httpclient.methods.PostMethod@4d54f318 failed, error
> code 500 and response:         {"url":"http://zeus001.jp:
> 7070/kylin/api/query","exception":"Error while executing SQL \"select
> \"DIM_DATE\".\"DATE_ID\" as \"c0\", \"V_DIM_CATE_LEVEL2\".\"CATE_LEVEL1_ID\"
> as \"c1\", \"V_DIM_BRAND\".\"BRAND_RANK\" as \"c2\",
> \"DIM_GOODS_TYPE\".\"GOODS_TYPE_ID\" as \"c3\",        count(distinct
> \"FCT_ORDR_PATH_OLAP\".\"GU_ID\") as \"m0\", count(distinct
> \"FCT_ORDR_PATH_OLAP\".\"PAY_USER_ID\") as \"m1\"         from
> \"FCT_ORDR_PATH_OLAP\" as \"FCT_ORDR_PATH_OLAP\" join \"DIM_DATE\" as
> \"DIM_DATE\" on \"FCT_ORDR_PATH_OLAP\".\"DATE_ID\" =
> \"DIM_DATE\".\"DATE_ID\" join \"V_DIM_CATE_LEVEL2\" as
> \"V_DIM_CATE_LEVEL2\"         on \"FCT_ORDR_PATH_OLAP\".\"CATE_LEVEL2_ID\"
> = \"V_DIM_CATE_LEVEL2\".\"CATE_LEVEL2_ID\" join \"V_DIM_BRAND\" as
> \"V_DIM_BRAND\" on \"FCT_ORDR_PATH_OLAP\".\"BRAND_ID\" =
> \"V_DIM_BRAND\".\"BRAND_ID\" join \"DIM_GOODS_TYPE\" as \"DIM_GOODS_TYPE\"
> on \"FCT_ORDR_PATH_OLAP\".\"GOODS_TYPE_ID\" = \"DIM_GOODS_TYPE\".\"GOODS_TYPE_ID\"
>        where \"DIM_DATE\".\"DATE_ID\" in ('2016-10-28', '2016-10-29',
> '2016-10-30', '2016-10-31')        group by \"DIM_DATE\".\"DATE_ID\",
> \"V_DIM_CATE_LEVEL2\".\"CATE_LEVEL1_ID\", \"V_DIM_BRAND\".\"BRAND_RANK\",
> \"DIM_GOODS_TYPE\".\"GOODS_TYPE_ID\"\":
>       <sub-thread for GTScanRequest 1d11f21d> The coprocessor thread
> stopped itself due to scan timeout or scan threshold(check region server
> log), failing current query..."}"height:nullleftOffset:0query:
> nullrowTotalsLists:nullruntime:nulltopOffset:0width:null
> and the HttpRequest only 11s,    and I set the hbase.rpc.timeout = 180s
> it's the same !
> I want Konw what the reason of the error ?
>
>     thanks!




-- 
Best regards,

Shaofeng Shi 史少锋

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message