From user-return-4252-archive-asf-public=cust-asf.ponee.io@kylin.apache.org Thu Apr 18 07:08:40 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 2D17C18061A for ; Thu, 18 Apr 2019 09:08:40 +0200 (CEST) Received: (qmail 35379 invoked by uid 500); 18 Apr 2019 07:08:39 -0000 Mailing-List: contact user-help@kylin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@kylin.apache.org Delivered-To: mailing list user@kylin.apache.org Received: (qmail 35369 invoked by uid 99); 18 Apr 2019 07:08:39 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Apr 2019 07:08:39 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id ADBE5C2850 for ; Thu, 18 Apr 2019 07:08:38 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.048 X-Spam-Level: ** X-Spam-Status: No, score=2.048 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id OYnjQDhlk_5s for ; Thu, 18 Apr 2019 07:08:37 +0000 (UTC) Received: from mail-it1-f195.google.com (mail-it1-f195.google.com [209.85.166.195]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 06DFF60E56 for ; Thu, 18 Apr 2019 07:08:37 +0000 (UTC) Received: by mail-it1-f195.google.com with SMTP id u65so1852410itc.2 for ; Thu, 18 Apr 2019 00:08:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=3Nw4t1fNma3FhPgMj/G6qr9+krGQ3uTDy79FBZLkmh4=; b=UtW0CkAeZ2zq5QsVNxMixcU+xhkdpv6L6xweh2LVaGX86VZF3xDOz9PSqZ8NsHTEPS Ruyk25KAgpZeKeRrroWtXFS0VUsmgRFK6MtJlGIgj7qVKA9WLnK9bl+MxmdIjF8KythB 2Z1a7YU2Qra+KkctsCGZ5NlTBIdWhAcIKw5AGOxlK+fX0XmNg8NpuVydolU1pFqIFxJi sdNyjgcgvaUSY0mz49r6kJaWETb+ZZRMtgJO01kMLR1CPtQdcfex6RivHPyszMyvAcDf WH8fjPdduMzA7OtnZZwMdM78duYDvH9nSBMqBj1y1AsuTWbT43ooisZ/HogMJhN5mslb tCfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=3Nw4t1fNma3FhPgMj/G6qr9+krGQ3uTDy79FBZLkmh4=; b=SzVh5rKixbRMvmEgCYtYbEm9pnIEguw7MPz+9BX+x00B17hJsypMkL7VCPW/6100rg 1Vds1HhNDpn4Zss1Ncd1vOOy9iQH4vCf0N2BeG+Ycd1m8Q58Qq/TRAqwpt8DJwD7Y8Ku 56kZ5WdXsrVOpG9Bp3g1qiHxc7xE8HyfDhrLIVNs203qOuUffOLRoV65RAmr8nLcWzIe jY5cL46g79WtULBVBol/n5lAqsKhw+oBrKia927UKzbk4zxTBU1+HihO8ne1+mvUsngP hpFOKhpHZtcjJb7VQi5wtf3sfuf46LFfaCY/UUqGLKqlAet1HlMppMWafudV6N/1MHaI aPcg== X-Gm-Message-State: APjAAAU/n9sG4K84tBi4PdRLM92dXkeOH/wWjD8KV8xnXngLkts5VeIl zb7r928mdQhZuSLmy0I+5UkHA9UDODTxWbGCN2TsgKy7xOg= X-Google-Smtp-Source: APXvYqwru1DtOPIk92k4EyA69RF6nWSVMrY6Z9CFxRKb6KEKpRPU+fjyGDM+NVGCRlkzahuzQtdjjBgH/w4v+aDVOsk= X-Received: by 2002:a24:e14b:: with SMTP id n72mr2052694ith.34.1555571316326; Thu, 18 Apr 2019 00:08:36 -0700 (PDT) MIME-Version: 1.0 References: <55607f03.d40.16a2e671c1c.Coremail.lk_hadoop@163.com> In-Reply-To: <55607f03.d40.16a2e671c1c.Coremail.lk_hadoop@163.com> From: Chao Long Date: Thu, 18 Apr 2019 15:08:24 +0800 Message-ID: Subject: Re: why kylin job failed when use beeline with zookeeper To: user@kylin.apache.org Content-Type: multipart/alternative; boundary="000000000000bb8ba90586c8ab87" --000000000000bb8ba90586c8ab87 Content-Type: text/plain; charset="UTF-8" Hi lk, First step use SSHClient to run "Create Hive Table" command, so I think it will not use beeline to connect hive. "Redistribute Flat Hive Table" step need to compute row count of flat table, so it will use beeline to connect if you configured. And I see the zookeeper connect string are enclosed in double quotes, is that a right way? On Thu, Apr 18, 2019 at 11:04 AM lk_hadoop wrote: > hi,all: > I'm using kylin-2.6.1-bin-cdh57 , when I connect to hive with > beelin : > kylin.source.hive.beeline-params=-n hive -p hiveadmin --hiveconf > hive.security.authorization.sqlstd.confwhitelist.append='mapreduce.job.*|dfs.*' > -u > jdbc:hive2://"bdp-scm-04:2181,bdp-scm-03:2181,bdp-scm-05:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2" > I got error at step : Redistribute Flat Hive Table > > > java.lang.IllegalArgumentException: Illegal character in path at index 86: hive2://dummyhost:00000/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2" > at java.net.URI.create(URI.java:852) > at org.apache.hive.jdbc.Utils.parseURL(Utils.java:302) > at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:122) > at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) > at java.sql.DriverManager.getConnection(DriverManager.java:664) > at java.sql.DriverManager.getConnection(DriverManager.java:208) > at org.apache.kylin.source.hive.BeelineHiveClient.init(BeelineHiveClient.java:72) > at org.apache.kylin.source.hive.BeelineHiveClient.(BeelineHiveClient.java:66) > at org.apache.kylin.source.hive.HiveClientFactory.getHiveClient(HiveClientFactory.java:29) > at org.apache.kylin.source.hive.RedistributeFlatHiveTableStep.computeRowCount(RedistributeFlatHiveTableStep.java:40) > at org.apache.kylin.source.hive.RedistributeFlatHiveTableStep.doWork(RedistributeFlatHiveTableStep.java:91) > at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166) > at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) > at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166) > at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.net.URISyntaxException: Illegal character in path at index 86: hive2://dummyhost:00000/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2" > at java.net.URI$Parser.fail(URI.java:2848) > > I don't know why , because I can pass the first step which also use > the same JDBC URL. > > 2019-04-18 > ------------------------------ > lk_hadoop > --000000000000bb8ba90586c8ab87 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi lk,
=C2=A0 First step use SSHClient to run "Cr= eate Hive Table" command, so I think it will not use beeline to connec= t hive.
=C2=A0 "Redistribute Flat Hive Table"=C2=A0step= need to compute row count of flat table, so it will use beeline to connect= if you configured.
=C2=A0 =C2=A0And I see the zookeeper connect = string are enclosed in double quotes, is that a right way?

<= div class=3D"gmail_quote">
On Thu, Apr= 18, 2019 at 11:04 AM lk_hadoop <lk= _hadoop@163.com> wrote:
hi,all:
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0I'm using kylin-2.6.1-bi= n-cdh57 ,=20 when I connect to hive with beelin :
=C2=A0kylin.source.hive.beeline-params= =3D-n hive -p=20 hiveadmin --hiveconf=20 hive.security.authorization.sqlstd.confwhitelist.append=3D'mapreduce.jo= b.*|dfs.*'=20 -u=20 jdbc:hive2://"bdp-scm-04:2181,bdp-scm-03:2181,bdp-scm-05:2181/;service= DiscoveryMode=3DzooKeeper;zooKeeperNamespace=3Dhiveserver2"
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 I got error at step : Redistribute=20 Flat Hive Table
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=20 
java.lan=
g.IllegalArgumentException: Illegal character in path at index 86: hive2://=
dummyhost:00000/;serviceDiscoveryMode=3DzooKeeper;zooKeeperNamespace=3Dhive=
server2"
	at java.net.URI.create(URI.java:852)
	at org.apache.hive.jdbc.Utils.parseURL(Utils.java:302)
	at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:12=
2)
	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:208)
	at org.apache.kylin.source.hive.BeelineHiveClient.init(BeelineHiveClient.j=
ava:72)
	at org.apache.kylin.source.hive.BeelineHiveClient.<init>(BeelineHive=
Client.java:66)
	at org.apache.kylin.source.hive.HiveClientFactory.getHiveClient(HiveClient=
Factory.java:29)
	at org.apache.kylin.source.hive.RedistributeFlatHiveTableStep.computeRowCo=
unt(RedistributeFlatHiveTableStep.java:40)
	at org.apache.kylin.source.hive.RedistributeFlatHiveTableStep.doWork(Redis=
tributeFlatHiveTableStep.java:91)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecu=
table.java:166)
	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultC=
hainedExecutable.java:71)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecu=
table.java:166)
	at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(Def=
aultScheduler.java:114)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja=
va:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j=
ava:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.URISyntaxException: Illegal character in path at index =
86: hive2://dummyhost:00000/;serviceDiscoveryMode=3DzooKeeper;zooKeeperName=
space=3Dhiveserver2"
	at java.net.URI$Parser.fail(URI.java:2848)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0I=20 don't know why , because I can pass the first step which also use the s= ame JDBC=20 URL.
=C2=A0=C2=A0=C2=A0=C2=A0
2019-04-18
lk_hadoop=C2= =A0
--000000000000bb8ba90586c8ab87--