kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: Kylin, EMR and HBase timeout
Date Thu, 08 Mar 2018 12:43:20 GMT
Hi Roberto,

Oh, I just saw you had already tried that. From the code level, I do say it
was set to the configuration when initiating a connection to HBase. Not
sure whether it was respected by the server or not.

Besides, these two parameters are very basic config, which cannot be set at
Cube level.

2018-03-08 19:29 GMT+08:00 <roberto.tardio@stratebi.com>:

> Hi,
>
>
>
> Could I override the following properties at cube definition?
>
>
>
> kylin.metadata.hbase-rpc-timeout=3600000
>
> kylin.metadata.hbase-client-scanner-timeout-period=3600000
>
>
>
> I would like to check if Kylin is using these values.
>
>
>
> I restarted kylin after modifying kylin.properties and also reload config.
> But looks like Kylins are not taking in to account these properties values.
>
>
>
> Kind Regards,
>
>
>
> *From:* roberto.tardio@stratebi.com [mailto:roberto.tardio@stratebi.com]
> *Sent:* jueves, 8 de marzo de 2018 11:33
> *To:* user@kylin.apache.org
> *Subject:* RE: Kylin, EMR and HBase timeout
>
>
>
> Hi,
>
>
>
> Thanks ShaoFeng!
>
>
>
> As a said on first email, I changed kylin.metadata.hbase-rpc-timeout=3600000
> in kylin.properties and restarted Kylin. However looks like that Kylin is
> not using this value, as the error log show “org.apache.hadoop.hbase.ipc.CallTimeoutException:
> Call id=4280915, waitTime=5001, *operationTimeout=5000*”
>
>
>
> About HBase Health:
>
>
>
> I executed command hbase hbck and looks like ok except the followinng:
>
>
>
> 1 inconsistencies detected.
>
> Status: INCONSISTENT
>
>
>
> Could this inconsistence affect HBase stability?I’m not an expert on HBase
> J.
>
>
>
> Best Regards,
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org
> <shaofengshi@apache.org>]
> *Sent:* jueves, 8 de marzo de 2018 10:05
> *To:* user <user@kylin.apache.org>
> *Subject:* Re: Kylin, EMR and HBase timeout
>
>
>
> followed the previous, the value is in milliseconds, the default is:
>
>
>
> *kylin.metadata.hbase-rpc-timeout=*5000
>
>
>
> 2018-03-08 17:03 GMT+08:00 ShaoFeng Shi <shaofengshi@apache.org>:
>
> Hi Roberto,
>
>
>
> Is the HBase cluster in health status? Is there any error/warning in
> region server's log? Suggest to take a look at the server side.
>
>
>
> The long timeout 3600000 value is to allow HBase transfer the Cube (HFile)
> to S3. It is not beautiful, but without it we may fail to build.
>
>
>
> The error you faced is when accessing Kylin metadata, the timeout value is
> overwritten to 5 seconds (with parameter "
> *kylin.metadata.hbase-rpc-timeout*"). You can customize it in
> kylin.properties if want.
>
>
>
>
>
>
>
> 2018-03-07 22:50 GMT+08:00 <roberto.tardio@stratebi.com>:
>
> Hi,
>
>
>
> We have a Kylin 2.2 installation that uses an EMR cluster, so it uses
> HBase on S3 as storage. It works well usually, but sometimes building
> process get error at step “Build Dimension Dictionary”, as hbase.ipc.
> callTimeoutException.
>
>
>
>
>
> In order to solve it, I changed rpc configuration as indicated in
> http://kylin.apache.org/docs23/install/kylin_aws_emr.html . I have
> changed, the following properties on hbase-site.xml for master and all
> slaves.
>
>
>
> <property>
>
>     <name>hbase.rpc.timeout</name>
>
>     <value>3600000</value>
>
>   </property>
>
>
>
>   <property>
>
>     <name>hbase.client.scanner.timeout.period</name>
>
>     <value>3600000</value>
>
>   </property>
>
>
>
> Then I restarted HBase Master and HBase Regions Servers. I checked
>
>
>
>
>
> However I still, getting the same error. Looks like that Kylin uses
> different value for these properties.
>
>
>
> I also changed the following kylin.properties:
>
>
>
> kylin.metadata.hbase-rpc-timeout=3600000
>
> kylin.metadata.hbase-client-scanner-timeout-period=3600000
>
>
>
> I restarted Kyling, but still getting same issue.
>
>
>
> Someone could help me?
>
>
>
> Thanks in advance!
>
> *Roberto Tardío Olmos*
>
> *Senior Big Data & Business Intelligence Consultant*
>
> Avenida de Brasil, 17
> <https://maps.google.com/?q=Avenida+de+Brasil,+17&entry=gmail&source=g>,
> Planta 16.28020 Madrid
>
> Fijo: 91.788.34.10
>
>
> [image:
> http://www.stratebi.com/image/layout_set_logo?img_id=21615&t=1486381163544]
>
>
>
> http://bigdata.stratebi.com/
>
>
>
> http://www.stratebi.com
>
>
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi 史少锋
>
>
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi 史少锋
>
>
>



-- 
Best regards,

Shaofeng Shi 史少锋

Mime
View raw message