kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 王廉鑫 <apache_...@163.com>
Subject Re: when cube run 4 step.it error.
Date Tue, 10 Sep 2019 10:04:56 GMT
it's case when kylin read orc format table,if the table is textfile format,is can work.


| |
王廉鑫
|
|
apache_dev@163.com
|
签名由网易邮箱大师定制
On 9/10/2019 17:25,Yaqian Zhang<Yaqian_Zhang@126.com> wrote:
Hi 廉鑫:
This artical maybe could resolve your problem.
https://stackoverflow.com/questions/35075276/apache-kylin-intermediate-table-not-found

On Sep 9, 2019, at 5:55 PM, 王廉鑫 <apache_dev@163.com> wrote:

Hi,




kylin2.6.2 on FusionInsight C.6.5.1




when Cbur run 4 Step. it get eror!!!


.
2019-09-09 17:51:31,538 INFO  [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511]
common.HCatUtil:638 : Configuration differences={hive.metastore.sasl.enabled=true, hive.metastore.try.direct.sql.ddl=false,
hive.metastore.kerberos.principal=hive/hadoop.hadoop.com@HADOOP.COM, javax.jdo.option.ConnectionURL=jdbc:postgresql://10.10.20.96:20051/hive${SERVICE_INDEX}meta?socketTimeout=60,
hive.server2.thrift.sasl.qop=auth-conf, hive.mapred.reduce.tasks.speculative.execution=false,
javax.jdo.option.ConnectionUserName=hive, hive.server2.authentication.kerberos.principal=hive/hadoop.hadoop.com@HADOOP.COM,
javax.jdo.option.ConnectionPassword=d2NjX2NyeXB0ATQxNDU1MzVGNDM0MjQzOzQxNDEzODQxNDEzNDQzMzkzNDM2Mzc0MTMyMzYzNTMzMzI0MjM4NDY0NDQyNDEzNDQzMzAzNjM4NDMzOTMxNDM7OzMyMzUzMDMwOzRFMjhDRkQyNTlDOTMxRTA5MjA2QUU2MzY2QzRCQzZBO0JFQTBDMzgyRDU1RkJGMzM7NjQzNjMxMzMzNDM3NjY2NTJEMzkzNTM1NjQyRDM0MzQzODM5MkQzOTYyMzczMzJENjU2NjM4MzU2MzMwMzgzMTM3MzY2NjYzOw,
hive.exec.scratchdir=/tmp/hive-scratch, javax.jdo.dbuser.decode.enable=true, hive.keychain.dir=/user/hive/keychain,
hive.metastore.metrics.enabled=true, hive.metastore.uris=thrift://10.10.20.94:21088,thrift://10.10.20.91:21088,
hive.server2.authentication=KERBEROS, javax.jdo.option.ConnectionDriverName=org.postgresql.Driver,
hive.server2.authentication.kerberos.keytab=/opt/huawei/Bigdata/FusionInsight_HD_6.5.1/install/FusionInsight-Hive-3.1.0/hive-3.1.0/bin/hive.keytab}
2019-09-09 17:51:31,540 INFO  [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511]
metastore.HiveMetaStoreClient:535 : Trying to connect to metastore with URI thrift://RDC-BD11:21088
2019-09-09 17:51:31,541 INFO  [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511]
metastore.HiveMetaStoreClient:608 : HMSC::open(): Could not find delegation token. Creating
KERBEROS-based thrift connection.
2019-09-09 17:51:31,547 INFO  [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511]
metastore.HiveMetaStoreClient:635 : Opened a connection to metastore, current connections:
1
2019-09-09 17:51:31,548 INFO  [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511]
metastore.HiveMetaStoreClient:740 : Connected to metastore.
2019-09-09 17:51:31,548 INFO  [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511]
metastore.RetryingMetaStoreClient:98 : RetryingMetaStoreClient proxy=class org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient
ugi=metric@HADOOP.COM (auth:KERBEROS) retries=1 delay=1 lifetime=0
2019-09-09 17:51:31,565 WARN  [Scheduler 1464751025 Job d780b707-aebf-802b-bf84-dc96805d8a10-367]
model.Segments:444 : NEW segment start does not fit/connect with other segments: GL1221160200CAMT[20190901000000_20190902000000]
2019-09-09 17:51:31,566 ERROR [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511]
common.MapReduceExecutable:138 : error execute MapReduceExecutable{id=23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-00,
name=null, state=RUNNING}
java.lang.RuntimeException: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT
table not found)
at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:83)
at org.apache.kylin.source.hive.cardinality.HiveColumnCardinalityJob.run(HiveColumnCardinalityJob.java:93)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:131)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT
table not found)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:80)
... 9 more
Caused by: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT
table not found)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result.read(ThriftHiveMetastore.java)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_req(ThriftHiveMetastore.java:2113)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_req(ThriftHiveMetastore.java:2100)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1753)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1745)
at sun.reflect.GeneratedMethodAccessor158.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:209)
at com.sun.proxy.$Proxy77.getTable(Unknown Source)
at org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:191)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:105)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:88)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
... 11 more
2019-09-09 17:51:31,566 WARN  [Scheduler 1464751025 Job d780b707-aebf-802b-bf84-dc96805d8a10-367]
model.Segments:446 : NEW segment end does not fit/connect with other segments: GL1221160200CAMT[20190901000000_20190902000000]
2019-09-09 17:51:31,583 ERROR [Scheduler 1464751025 Job 23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b-511]
execution.AbstractExecutable:169 : error running Executable: DefaultChainedExecutable{id=23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b,
name=Hive Column Cardinality calculation for table 'METRIC_V2_AIA.GL1221160200CAMT', state=RUNNING}
2019-09-09 17:51:31,600 ERROR [pool-12-thread-10] threadpool.DefaultScheduler:116 : ExecuteException
job:23bc04b4-b99d-fa4c-97db-a0b4ad0e1d2b
org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.exception.ExecuteException:
java.lang.RuntimeException: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT
table not found)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:182)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kylin.job.exception.ExecuteException: java.lang.RuntimeException: java.io.IOException:
NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT
table not found)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:182)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
... 4 more
Caused by: java.lang.RuntimeException: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT
table not found)
at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:83)
at org.apache.kylin.source.hive.cardinality.HiveColumnCardinalityJob.run(HiveColumnCardinalityJob.java:93)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:131)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
... 6 more
Caused by: java.io.IOException: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT
table not found)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:80)
... 9 more
Caused by: NoSuchObjectException(message:hive.DEFAULT.KYLIN_INTERMEDIATE_METRIC_V2_AIA_GL1221160200CAMT
table not found)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result.read(ThriftHiveMetastore.java)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_req(ThriftHiveMetastore.java:2113)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_req(ThriftHiveMetastore.java:2100)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1753)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1745)
at sun.reflect.GeneratedMethodAccessor158.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:209)
at com.sun.proxy.$Proxy77.getTable(Unknown Source)
at org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:191)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:105)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:88)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
... 11 more
2019-09-09 17:51:31,652 WARN  [Scheduler 1464751025 Job d780b707-aebf-802b-bf84-dc96805d8a10-367]
model.Segments:444 : NEW segment start does not fit/connect with other segments: GL1221160200CAMT[20190901000000_20190902000000]








|
apache_dev@163.com
|
签名由网易邮箱大师定制
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message