Mailing-List: contact issues-help@kylin.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kylin.apache.org
Date: Mon, 13 Nov 2017 09:31:00 +0000 (UTC)
From: "Shaofeng SHI (JIRA)" <jira@apache.org>
To: issues@kylin.apache.org
Message-ID: <JIRA.13117858.1510559588000.219725.1510565460242@Atlassian.JIRA>
In-Reply-To: <JIRA.13117858.1510559588000@Atlassian.JIRA>
References: <JIRA.13117858.1510559588000@Atlassian.JIRA> <JIRA.13117858.1510559588092@jira-lw-us.apache.org>
Subject: [jira] [Commented] (KYLIN-3035) How to use Kylin on EMR with S3 as
 hbase storage
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Mon, 13 Nov 2017 09:31:06 -0000


    [ https://issues.apache.org/jira/browse/KYLIN-3035?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1624=
9291#comment-16249291 ]=20

Shaofeng SHI commented on KYLIN-3035:
-------------------------------------

Hi Shawn, here is a reference about how to use S3 as the storage for Kylin:=
 https://issues.apache.org/jira/browse/KYLIN-3028

From your error, it seems Kylin couldn't connect with the HBase server. ple=
ase check:
1) HBase is running well; you can check this with hbase shell, to see wheth=
er Kylin metadata table is active and can be scanned.
2) Check whether you have updated the "hbase.zookeeper.quorum" properties i=
n conf/kylin_job_conf.xml, or the hbase-site.xml in your Kylin machine is u=
pdated with the new cluster.


> How to use Kylin on EMR with S3 as hbase storage
> ------------------------------------------------
>
>                 Key: KYLIN-3035
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3035
>             Project: Kylin
>          Issue Type: Bug
>          Components: Metadata
>    Affects Versions: v2.2.0
>         Environment: EMR 5.5.0, Kylin 2.2.0
>            Reporter: Shawn Wang
>            Assignee: Shaofeng SHI
>
> Can somebody give an example of how to use kylin on EMR with S3 as hbase =
storage, which support reuse the previously built cube on new EMR after the=
 original EMR has been terminated.
> My purpose is simple:
> 1. use transient EMR cluster to build cubes
> 2. use a persistent cluster to handle query requests
> Of course, the culsters should share same hbase storage, so I setup the c=
luster to use S3 as hbase storage, after 2.2.0 fix the "HFile not written t=
o S3" issue, I have been built a sample cube successfully, using configurat=
ions:
> EMR:
> {noformat}
> [
> =09{
> =09=09"Classification": "hbase-site",
> =09=09"Properties": {
> =09=09=09"hbase.rootdir": "s3://kylin-emrfs/hbase-production"
> =09=09}
> =09},
> =09{
> =09=09"Classification": "hbase",
> =09=09"Properties": {
> =09=09=09"hbase.emr.storageMode": "s3"
> =09=09}
> =09},
> =09{
> =09=09"Classification": "emrfs-site",
> =09=09"Properties": {
> =09=09=09"fs.s3.consistent": "true",
> =09=09=09"fs.s3.consistent.metadata.tableName": "KylinEmrFSMetadata"
> =09=09}
> =09}
> ]
> {noformat}
> kylin.propertities:
> {noformat}
> kylin.env.hdfs-working-dir=3Ds3://kylin/kylin-emrfs/kylin-working-dir
> kylin.server.mode=3Dall
> {noformat}
> Then I create a new cluster with same EMR configuration and query mode fo=
r kylin, kylin just can't startup with errors:
> {noformat}
> 2017-11-13 07:33:44,415 INFO  [main-SendThread(ip-172-31-1-10.cn-north-1.=
compute.internal:2181)] zookeeper.ClientCnxn:876 : Socket connection establ=
ished to ip-172-31-1-10.cn-north-1.compute.internal/172.31.1.10:2181, initi=
ating session
> 2017-11-13 07:33:44,422 INFO  [main-SendThread(ip-172-31-1-10.cn-north-1.=
compute.internal:2181)] zookeeper.ClientCnxn:1299 : Session establishment c=
omplete on server ip-172-31-1-10.cn-north-1.compute.internal/172.31.1.10:21=
81, sessionid =3D 0x15fb4173c100156, negotiated timeout =3D 40000
> 2017-11-13 07:33:48,380 DEBUG [main] hbase.HBaseConnection:279 : HTable '=
kylin_metadata' already exists
> Exception in thread "main" java.lang.IllegalArgumentException: Failed to =
find metadata store by url: kylin_metadata@hbase
> =09at org.apache.kylin.common.persistence.ResourceStore.createResourceSto=
re(ResourceStore.java:89)
> =09at org.apache.kylin.common.persistence.ResourceStore.getStore(Resource=
Store.java:101)
> =09at org.apache.kylin.rest.service.AclTableMigrationTool.checkIfNeedMigr=
ate(AclTableMigrationTool.java:94)
> =09at org.apache.kylin.tool.AclTableMigrationCLI.main(AclTableMigrationCL=
I.java:41)
> Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Fail=
ed after attempts=3D1, exceptions:
> Mon Nov 13 07:33:48 UTC 2017, RpcRetryingCaller{globalStartTime=3D1510558=
428667, pause=3D100, retries=3D1}, java.net.ConnectException: =E6=8B=92=E7=
=BB=9D=E8=BF=9E=E6=8E=A5
> =09at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(Rp=
cRetryingCaller.java:159)
> =09at org.apache.hadoop.hbase.client.HTable.get(HTable.java:864)
> =09at org.apache.hadoop.hbase.client.HTable.get(HTable.java:830)
> =09at org.apache.kylin.storage.hbase.HBaseResourceStore.internalGetFromHT=
able(HBaseResourceStore.java:385)
> =09at org.apache.kylin.storage.hbase.HBaseResourceStore.getFromHTable(HBa=
seResourceStore.java:363)
> =09at org.apache.kylin.storage.hbase.HBaseResourceStore.existsImpl(HBaseR=
esourceStore.java:116)
> =09at org.apache.kylin.common.persistence.ResourceStore.exists(ResourceSt=
ore.java:144)
> =09at org.apache.kylin.common.persistence.ResourceStore.createResourceSto=
re(ResourceStore.java:84)
> =09... 3 more
> Caused by: java.net.ConnectException: =E6=8B=92=E7=BB=9D=E8=BF=9E=E6=8E=
=A5
> =09at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> =09at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:7=
17)
> =09at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeo=
ut.java:206)
> =09at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
> =09at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
> =09at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupConnectio=
n(RpcClientImpl.java:416)
> =09at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams=
(RpcClientImpl.java:722)
> =09at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(R=
pcClientImpl.java:909)
> =09at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteReq=
uest(RpcClientImpl.java:873)
> =09at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1=
244)
> =09at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(Ab=
stractRpcClient.java:227)
> =09at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImp=
lementation.callBlockingMethod(AbstractRpcClient.java:336)
> =09at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientServi=
ce$BlockingStub.get(ClientProtos.java:35372)
> =09at org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:856)
> =09at org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:847)
> =09at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(Rp=
cRetryingCaller.java:137)
> =09... 10 more
> 2017-11-13 07:33:48,709 INFO  [Thread-1] client.ConnectionManager$HConnec=
tionImplementation:2180 : Closing master protocol: MasterService
> 2017-11-13 07:33:48,710 INFO  [Thread-1] client.ConnectionManager$HConnec=
tionImplementation:1718 : Closing zookeeper sessionid=3D0x15fb4173c100156
> 2017-11-13 07:33:48,712 INFO  [Thread-1] zookeeper.ZooKeeper:684 : Sessio=
n: 0x15fb4173c100156 closed
> 2017-11-13 07:33:48,712 INFO  [main-EventThread] zookeeper.ClientCnxn:519=
 : EventThread shut down for session: 0x15fb4173c100156
> ERROR: Unknown error. Please check full log.
> {noformat}
> And if I change the Kylin server mode to all, Kylin can startup, but the =
page on port 7070 can not be opened, with similar errors as above.
> I am wondering If there are some other configuration I have missed, or I =
am just in the wrong way.
> It'll be many thanks if someone can give an complete example to show how =
to get this work done!


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)