kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: intermediate table not found when used the hive view
Date Tue, 18 Jul 2017 09:48:30 GMT
Let me confirm one thing; the deleted hive table is an intermediate table
for a view, "kylin_intermediate_DW_ZYB_V_DIM_SCENIC" in this case. The view
"DW_ZYB_V_DIM_SCENIC" is a lookup table.

This is a know issue; It impacts on using View as lookup, and run multiple
builds in parallel. I had thought its occurancy will be very low, that's my
fault. Please open a JIRA to track this.

A not-perfect solution is, use different views for different cubes. But
that may impact on query patterns. Another suggestion would be, using
normal hive table as lookup, that doesn't have this issue.

2017-07-18 15:22 GMT+08:00 jianhui.yi <jianhui.yi@zhiyoubao.com>:

> Hi all,
>
> I used the Hive view to build the cube, and multiple cube all use the same
> view. When the build task is submitted at the same time, there will be an
> error in the step "Extract Fact Table Distinct Columns".
>
> I noticed that the previous build task will delete the table in the "Create
> Intermediate Flat Hive Table" step, which should be the next task of
> building the cause of the error, how to solve this problem?
>
> Thanks!
>
>
>
> 2017-07-17 08:58:49,803 ERROR [Job 71a3e0e4-a713-44f8-a473-
> 7496245857ec-302]
> common.HadoopShellExecutable:65 : error execute
> HadoopShellExecutable{id=71a3e0e4-a713-44f8-a473-7496245857ec-04, name=
>
> Build Dimension Dictionary, state=RUNNING}
>
> java.lang.RuntimeException: cannot get HiveTableMeta
>
>         at org.apache.kylin.source.hive.HiveTable.<init>(HiveTable.
> java:50)
>
>         at
> org.apache.kylin.source.hive.HiveSource.createReadableTable(
> HiveSource.java:
> 49)
>
>         at
> org.apache.kylin.source.SourceFactory.createReadableTable(
> SourceFactory.java
> :42)
>
>         at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.
> decideInputTable(Dictionary
> GeneratorCLI.java:118)
>
>         at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.
> processSegment(DictionaryGe
> neratorCLI.java:63)
>
>         at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.
> processSegment(DictionaryGe
> neratorCLI.java:54)
>
>         at
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(
> CreateDictionaryJob
> .java:66)
>
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>
>         at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.
> doWork(HadoopShellEx
> ecutable.java:63)
>
>         at
> org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable
> .java:124)
>
>         at
> org.apache.kylin.job.execution.DefaultChainedExecutable.
> doWork(DefaultChaine
> dExecutable.java:64)
>
>         at
> org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable
> .java:124)
>
>         at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$
> JobRunner.run(DefaultS
> cheduler.java:142)
>
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:11
> 45)
>
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:6
> 15)
>
>         at java.lang.Thread.run(Thread.java:745)
>
> Caused by: org.apache.hive.service.cli.HiveSQLException: Error while
> compiling statement: FAILED: SemanticException [Error 10001]: Table not
> found kylin_intermediate_DW_ZYB_V_DIM_SCENIC
>
>         at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:239)
>
>         at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.
> java:225)
>
>         at
> org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:244)
>
>         at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:375)
>
>         at
> org.apache.kylin.source.hive.BeelineHiveClient.
> getHiveTableMeta(BeelineHiveC
> lient.java:141)
>
>         at org.apache.kylin.source.hive.HiveTable.<init>(HiveTable.
> java:48)
>
>         ... 16 more
>
> Caused by: org.apache.hive.service.cli.HiveSQLException: Error while
> compiling statement: FAILED: SemanticException [Error 10001]: Table not
> found kylin_intermediate_DW_ZYB_V_DIM_SCENIC
>
>         at
> org.apache.hive.service.cli.operation.Operation.
> toSQLException(Operation.jav
> a:385)
>
>         at
> org.apache.hive.service.cli.operation.SQLOperation.
> prepare(SQLOperation.java
> :136)
>
>         at
> org.apache.hive.service.cli.operation.SQLOperation.
> runInternal(SQLOperation.
> java:206)
>
>         at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:327)
>
>         at
> org.apache.hive.service.cli.session.HiveSessionImpl.
> executeStatementInternal
> (HiveSessionImpl.java:424)
>
>         at
> org.apache.hive.service.cli.session.HiveSessionImpl.
> executeStatementAsync(Hi
> veSessionImpl.java:401)
>
>         at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl
> .java:43)
>
>         at java.lang.reflect.Method.invoke(Method.java:606)
>
>         at
> org.apache.hive.service.cli.session.HiveSessionProxy.
> invoke(HiveSessionProxy
> .java:78)
>
>         at
> org.apache.hive.service.cli.session.HiveSessionProxy.
> access$000(HiveSessionP
> roxy.java:36)
>
>         at
> org.apache.hive.service.cli.session.HiveSessionProxy$1.
> run(HiveSessionProxy.
> java:63)
>
>         at java.security.AccessController.doPrivileged(Native Method)
>
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.ja
> va:1783)
>
>         at
> org.apache.hive.service.cli.session.HiveSessionProxy.
> invoke(HiveSessionProxy
> .java:59)
>
>         at com.sun.proxy.$Proxy27.executeStatementAsync(Unknown Source)
>
>         at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(
> CLIService.java
> :260)
>
>         at
> org.apache.hive.service.cli.thrift.ThriftCLIService.
> ExecuteStatement(ThriftC
> LIService.java:500)
>
>         at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$
> ExecuteStatement.ge
> tResult(TCLIService.java:1313)
>
>         at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$
> ExecuteStatement.ge
> tResult(TCLIService.java:1298)
>
>         at
> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>
>         at org.apache.thrift.TBaseProcessor.process(
> TBaseProcessor.java:39)
>
>         at
> org.apache.hive.service.auth.TSetIpAddressProcessor.
> process(TSetIpAddressPro
> cessor.java:56)
>
>         at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(
> TThreadPoolServ
> er.java:286)
>
>         ... 3 more
>
> Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: Table not
> found kylin_intermediate_DW_ZYB_V_DIM_SCENIC
>
>         at
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer$
> QualifiedNameUtil.getTab
> leName(DDLSemanticAnalyzer.java:1835)
>
>         at
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.
> analyzeDescribeTable(DDL
> SemanticAnalyzer.java:1985)
>
>         at
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.
> analyzeInternal(DDLSeman
> ticAnalyzer.java:316)
>
>         at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.
> analyze(BaseSemanticAna
> lyzer.java:223)
>
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:488)
>
>         at
> org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1274)
>
>         at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1261)
>
>         at
> org.apache.hive.service.cli.operation.SQLOperation.
> prepare(SQLOperation.java
> :134)
>
>         ... 26 more
>
>


-- 
Best regards,

Shaofeng Shi 史少锋

Mime
View raw message