From dev-return-15910-archive-asf-public=cust-asf.ponee.io@kylin.apache.org Wed Jul 10 09:04:52 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id A632718064A for ; Wed, 10 Jul 2019 11:04:52 +0200 (CEST) Received: (qmail 81517 invoked by uid 500); 10 Jul 2019 09:04:51 -0000 Mailing-List: contact dev-help@kylin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kylin.apache.org Delivered-To: mailing list dev@kylin.apache.org Received: (qmail 81505 invoked by uid 99); 10 Jul 2019 09:04:51 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Jul 2019 09:04:51 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 8D1B91A34B6 for ; Wed, 10 Jul 2019 09:04:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.345 X-Spam-Level: *** X-Spam-Status: No, score=3.345 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, PDS_NO_HELO_DNS=1.294, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id gD3mYaOxfpJx for ; Wed, 10 Jul 2019 09:04:48 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::d44; helo=mail-io1-xd44.google.com; envelope-from=chao.long0101@gmail.com; receiver= Received: from mail-io1-xd44.google.com (mail-io1-xd44.google.com [IPv6:2607:f8b0:4864:20::d44]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id E9F447E20C for ; Wed, 10 Jul 2019 09:04:47 +0000 (UTC) Received: by mail-io1-xd44.google.com with SMTP id m24so3151386ioo.2 for ; Wed, 10 Jul 2019 02:04:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=t/xkQrVds11Ba57r30UvwUkwKt/BaxY7LJtGaO8t9uE=; b=a9OB3hwWDDR4Vl0RWTP7vH/uNvDL9y57x55Tt24UFjvaWIA4/EVa+fwqN+E4v2ias0 +uehQLDrdvn02HNQoMFH9i/FfhrJtyO24/cejs0lGPelQ3cq1DGEGC+D8/geW9vTacUu irB9ac1lOaOUO7i9e82G3POhergIEUpFcAJTTZDZjZdEpgVouA7WXw1vwCtqbSssioSn PmBAC84dWQekV6W2iQ58mz7emG/Y4gMsD+KM4mp81iXayQr+PyJ88odrs4zJSS5u/FTa EOpVLTeBmA4n2RQoeVyHWMed/i1jhV3fkGtBE2oVsG1Nl7GFAeRn6YacMokkoBC50oSY PhaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=t/xkQrVds11Ba57r30UvwUkwKt/BaxY7LJtGaO8t9uE=; b=iYh3lqte7wgZBZB6WsiCXYKIY7pU3KzJQ/VT9DgnnkzAT78+iLU386LJMZU2EuTnnM FKk8PbkvscPW/PbeYll9jB4u3l2zhL4Skf9p+58H7ZHi/jNbfATZRkszWElCV29IhkdK A8nYaCRTesS2xmutdf63waL76gVRNq0hs0vlX4KzUdmV5xOSle4Gg7xjVSPOj/Vatndk jrxr/ighVInvQ6KbJdtNzclUb7HhBlGLn8wCU3GoSoXqs88n9/57WtIrpFU0dMjSwYc3 e7bC6xvoWYf04V2My3Wej7AEYk97Pehcz3wrIAvV0JZs8eAxS6itUs6S6Cc9l0pVgrOQ UT3A== X-Gm-Message-State: APjAAAXz/IKsljYA0LpeVScNWMvptdsZdKxZLjoUM3yW/AmGGuJNsXQq o0HvZ1CdtVj8alC3Eh2xG2Gj7zxkN+Iz6yq3k6typoCAKaY= X-Google-Smtp-Source: APXvYqw0CvxqZthCaLh7ItHXVnzbV3X9AGwHZ9WqrgOOMIp6anuzm4+dF+UUS0fG9fZ79H1U6N5NeFk01Iu/VFQhIhE= X-Received: by 2002:a5d:834f:: with SMTP id q15mr9835977ior.59.1562749486543; Wed, 10 Jul 2019 02:04:46 -0700 (PDT) MIME-Version: 1.0 References: <61e2814d.ce44.16bd64a8f40.Coremail.wangzy24@163.com> <7e14f1f2.d01a.16bd653cc39.Coremail.wangzy24@163.com> <6c19162d.b3c3.16bdb090217.Coremail.wangzy24@163.com> In-Reply-To: <6c19162d.b3c3.16bdb090217.Coremail.wangzy24@163.com> From: Chao Long Date: Wed, 10 Jul 2019 17:04:35 +0800 Message-ID: Subject: Re: Re: cannot get HiveTableMeta To: dev@kylin.apache.org Content-Type: multipart/alternative; boundary="00000000000004bc35058d4ff8c3" --00000000000004bc35058d4ff8c3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable This is a known issue and have been fixed in https://issues.apache.org/jira/browse/KYLIN-3515. You can upgrade your Kylin to v2.6.0 or above. On Wed, Jul 10, 2019 at 4:39 PM =E6=9B=BC wrote: > The Kylin version is 1.6 > > > At 2019-07-10 10:36:03, "Chao Long" wrote: > >Hi > > What's your Kylin version? > > > >On Tue, Jul 9, 2019 at 6:42 PM =E6=9B=BC wrote: > > > >> There are two cubes using the same dimension table, and when both cube= s > >> are built at the same time, both will do delete and create operations, > >> causing one of them to fail, Please see the log > >> pool-9-thread-10 and pool-9-thread-9 are two cube builds that are > executed > >> separately, pool-9-thread-10 failed > >> > >> > >> 2019-07-09 15:49:03,870 INFO [pool-9-thread-10] > >> common.ShellExecutable:52 : executing:hive -e "USE kylin_flat_db; > >> SET dfs.replication=3D2; > >> SET hive.exec.compress.output=3Dtrue; > >> SET hive.auto.convert.join.noconditionaltask=3Dtrue; > >> SET hive.auto.convert.join.noconditionaltask.size=3D100000000; > >> SET mapreduce.output.fileoutputformat.compress.type=3DBLOCK; > >> SET mapreduce.job.split.metainfo.maxsize=3D-1; > >> > >> > >> DROP TABLE IF EXISTS > >> kylin_intermediate_EDW_D_EDW_HOTEL_PROFILE_INFO_FIN_ADD_INDEX_VIEW; > >> CREATE TABLE IF NOT EXISTS > >> kylin_intermediate_EDW_D_EDW_HOTEL_PROFILE_INFO_FIN_ADD_INDEX_VIEW > >> LOCATION > >> > '/kylin/kylin_metadata/kylin-88e0f1f2-c55e-44a2-88a9-37be8e07f445/kylin_i= ntermediate_EDW_D_EDW_HOTEL_PROFILE_INFO_FIN_ADD_INDEX_VIEW' > >> AS SELECT * FROM EDW.D_EDW_HOTEL_PROFILE_INFO_FIN_ADD_INDEX_VIEW; > >> > >> > >> " > >> > >> > >> > >> > >> > >> > >> 2019-07-09 15:49:53,997 INFO [pool-9-thread-9] > >> common.ShellExecutable:52 : executing:hive -e "USE kylin_flat_db; > >> SET dfs.replication=3D2; > >> SET hive.exec.compress.output=3Dtrue; > >> SET hive.auto.convert.join.noconditionaltask=3Dtrue; > >> SET hive.auto.convert.join.noconditionaltask.size=3D100000000; > >> SET mapreduce.output.fileoutputformat.compress.type=3DBLOCK; > >> SET mapreduce.job.split.metainfo.maxsize=3D-1; > >> > >> > >> DROP TABLE IF EXISTS > >> kylin_intermediate_EDW_D_EDW_HOTEL_PROFILE_INFO_FIN_ADD_INDEX_VIEW; > >> CREATE TABLE IF NOT EXISTS > >> kylin_intermediate_EDW_D_EDW_HOTEL_PROFILE_INFO_FIN_ADD_INDEX_VIEW > >> LOCATION > >> > '/kylin/kylin_metadata/kylin-feb1f455-3fa1-4d14-ba7c-b5d5fb9969b4/kylin_i= ntermediate_EDW_D_EDW_HOTEL_PROFILE_INFO_FIN_ADD_INDEX_VIEW' > >> AS SELECT * FROM EDW.D_EDW_HOTEL_PROFILE_INFO_FIN_ADD_INDEX_VIEW; > >> > >> > >> " > >> > >> > >> 2019-07-09 15:50:07,991 ERROR [pool-9-thread-10] > >> common.HadoopShellExecutable:65 : error execute > >> HadoopShellExecutable{id=3D88e0f1f2-c55e-44a2-88a9-37be8e07f445-04, > >> name=3DBuild Dimension Dicti > >> onary, state=3DRUNNING} > >> java.lang.RuntimeException: cannot get HiveTableMeta > >> at > org.apache.kylin.source.hive.HiveTable.(HiveTable.java:49) > >> at > >> > org.apache.kylin.source.hive.HiveSource.createReadableTable(HiveSource.ja= va:49) > >> at > >> > org.apache.kylin.source.SourceFactory.createReadableTable(SourceFactory.j= ava:42) > >> at > >> > org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:269= ) > >> at > >> > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(Dictionar= yGeneratorCLI.java:58) > >> at > >> > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(Dictionar= yGeneratorCLI.java:41) > >> at org.apache.kylin.engine.mr > >> .steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54) > >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > >> at org.apache.kylin.engine.mr > >> .common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > >> at > >> > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecuta= ble.java:113) > >> at > >> > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultCha= inedExecutable.java:57) > >> at > >> > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecuta= ble.java:113) > >> at > >> > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(Defau= ltScheduler.java:136) > >> at > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java= :1145) > >> at > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav= a:615) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: > >> > NoSuchObjectException(message:KYLIN_FLAT_DB.kylin_intermediate_EDW_D_EDW_= HOTEL_PROFILE_INFO_FIN_ADD_INDEX_VIEW > >> table not found) > >> at > >> > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result= $get_table_resultStandardScheme.read(ThriftHiveMetastore.java:35066) > >> at > >> > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result= $get_table_resultStandardScheme.read(ThriftHiveMetastore.java:35034) > >> at > >> > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result= .read(ThriftHiveMetastore.java:34965) > >> at > >> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) > >> at > >> > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_= table(ThriftHiveMetastore.java:1218) > >> at > >> > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table= (ThriftHiveMetastore.java:1204) > >> at > >> > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaSto= reClient.java:1209) > >> at > >> > org.apache.kylin.source.hive.CLIHiveClient.getHiveTableMeta(CLIHiveClient= .java:79) > >> at > org.apache.kylin.source.hive.HiveTable.(HiveTable.java:47) > >> ... 16 more > >> > >> > --00000000000004bc35058d4ff8c3--