kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xiang zheng <zhengxian...@gmail.com>
Subject Fwd: 1.6-cdh5.7版本,kylin构建cube时报Failed to create dictionary on xxxx.GROUP_ID,Caused by: com.google.common.cache.CacheLoader$InvalidCacheLoadException: CacheLoader returned null for key 3271559448516296710.
Date Sun, 07 Jul 2019 10:44:58 GMT
---------- Forwarded message ---------
发件人: xiang zheng <zhengxiangpx@gmail.com>
Date: 2019年7月7日周日 下午6:26
Subject: 1.6-cdh5.7版本,kylin构建cube时报Failed to create dictionary on
xxxx.GROUP_ID,Caused by:
com.google.common.cache.CacheLoader$InvalidCacheLoadException: CacheLoader
returned null for key 3271559448516296710.
To: <user@kylin.apache.org>


hi :
请教一个问题:
在1.6-cdh5.7版本,构建cube时,遇到了一个报错:

 GROUP_ID 是一个全局字典。度量是 count_distinct

Step 4 - Build Dimension Dictionary:

java.lang.RuntimeException: Failed to create dictionary on xxxx.GROUP_ID

 at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:325)

 at org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:222)

 at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)

 at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41)

 at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54
)

 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

 at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63
)

 at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)

 at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)

 at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)

 at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)

 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

 at java.lang.Thread.run(Thread.java:748)

Caused by: com.google.common.cache.CacheLoader$InvalidCacheLoadException:
CacheLoader returned null for key 3271559448516296710.

 at
com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2412)

 at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2380)

 at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)

 at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2257)

 at com.google.common.cache.LocalCache.get(LocalCache.java:4000)

 at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4004)

 at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874)

 at org.apache.kylin.dict.CachedTreeMap.get(CachedTreeMap.java:292)

 at org.apache.kylin.dict.CachedTreeMap.get(CachedTreeMap.java:52)

 at
org.apache.kylin.dict.AppendTrieDictionary$Builder.addValue(AppendTrieDictionary.java:821)

 at
org.apache.kylin.dict.AppendTrieDictionary$Builder.addValue(AppendTrieDictionary.java:804)

 at
org.apache.kylin.dict.GlobalDictionaryBuilder.build(GlobalDictionaryBuilder.java:78)

 at
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:81)

 at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:323)

 ... 14 more


kylin.log日志

2019-07-06 13:31:45,836 ERROR [pool-9-thread-11] dict.CachedTreeMap:256 :
read value from
/kylin/kylin_metadata/resources/GlobalDict/dict/库.表/GROUP_ID.tmp/cached_3271559448516296710
exception: java.io.FileNotFoundException: File does not exist:
/kylin/kylin_metadata/resources/GlobalDict/di
ct/库.表/GROUP_ID.tmp/cached_3271559448516296710
        at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
        at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
        at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:2092)
        at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2062)
        at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1975)
        at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:575)
        at
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java
:92)
        at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.j
ava:376)
        at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.jav
a)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2222)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)

java.io.FileNotFoundException: File does not exist:
/kylin/kylin_metadata/resources/GlobalDict/dict/库.表/GROUP_ID.tmp/cached_327155944851
6296710
        at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
        at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
        at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:2092)
        at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2062)
        at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1975)
        at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:575)
        at
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java
:92)
        at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.j
ava:376)
        at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.jav
a)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2222)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)

       at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
       at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
       at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
       at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
       at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
       at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
       at
org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1289)
       at
org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1274)
       at
org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1262)
       at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:307)
       at
org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:273)
       at
org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:265)
       at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1593)
       at
org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:338)
       at
org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:334)
       at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
       at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:334)
       at
org.apache.kylin.dict.CachedTreeMap.readValue(CachedTreeMap.java:251)
       at
org.apache.kylin.dict.CachedTreeMap.access$500(CachedTreeMap.java:52)
       at org.apache.kylin.dict.CachedTreeMap$2.load(CachedTreeMap.java:177)
       at org.apache.kylin.dict.CachedTreeMap$2.load(CachedTreeMap.java:174)
       at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
       at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)
       at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)
       at
com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2257)
       at com.google.common.cache.LocalCache.get(LocalCache.java:4000)
       at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4004)
       at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874)
       at org.apache.kylin.dict.CachedTreeMap.get(CachedTreeMap.java:292)
       at org.apache.kylin.dict.CachedTreeMap.get(CachedTreeMap.java:52)
       at
org.apache.kylin.dict.AppendTrieDictionary$Builder.addValue(AppendTrieDictionary.java:821)
       at
org.apache.kylin.dict.AppendTrieDictionary$Builder.addValue(AppendTrieDictionary.java:804)
       at
org.apache.kylin.dict.GlobalDictionaryBuilder.build(GlobalDictionaryBuilder.java:78)
       at
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:81)
        at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:323)
        at
org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:222)
        at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)
        at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41)
        at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
        at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
        at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by:
org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): File
does not exist:
/kylin/kylin_metadata/resources/GlobalDict/dict/库.表/GROUP_ID.tmp/cached_3271559448516296710
        at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
        at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
        at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:2092)
        at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2062)
        at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1975)
        at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:575)
        at
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java
:92)
        at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.j
ava:376)
        at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.jav
a)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2222)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)

        at org.apache.hadoop.ipc.Client.call(Client.java:1504)
        at org.apache.hadoop.ipc.Client.call(Client.java:1441)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
        at com.sun.proxy.$Proxy28.getBlockLocations(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:260)
       at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
       at com.sun.proxy.$Proxy29.getBlockLocations(Unknown Source)
       at
org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1287)
       ... 42 more
2019-07-06 13:31:45,836 INFO  [pool-9-thread-11] dict.CachedTreeMap:178 :
Load cache by key 3271559448516296710(-1828817368) with value null
2019-07-06 13:31:45,838 ERROR [pool-9-thread-11]
common.HadoopShellExecutable:65 : error execute
HadoopShellExecutable{id=abec6f31-ec08-4338-a673-fe2f79f8
0ec4-03, name=Build Dimension Dictionary, state=RUNNING}
java.lang.RuntimeException: Failed to create dictionary on 库.表.GROUP_ID
       at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:325)
       at
org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:222)
       at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)
       at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41)
       at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
       at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
       at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
       at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
       at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
       at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
       at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.common.cache.CacheLoader$InvalidCacheLoadException:
CacheLoader returned null for key 3271559448516296710.
       at
com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2412)
       at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2380)
       at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)
       at
com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2257)
       at com.google.common.cache.LocalCache.get(LocalCache.java:4000)
       at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4004)
       at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874)
       at org.apache.kylin.dict.CachedTreeMap.get(CachedTreeMap.java:292)
       at org.apache.kylin.dict.CachedTreeMap.get(CachedTreeMap.java:52)
       at
org.apache.kylin.dict.AppendTrieDictionary$Builder.addValue(AppendTrieDictionary.java:821)
       at
org.apache.kylin.dict.AppendTrieDictionary$Builder.addValue(AppendTrieDictionary.java:804)
       at
org.apache.kylin.dict.GlobalDictionaryBuilder.build(GlobalDictionaryBuilder.java:78)
       at
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:81)
       at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:323)
       ... 14 more



result code:2

在对应的kylin的元数据目录下没有找到对应的3271559448516296710
cache文件,也自己不能创建。每次build都去找这个3271559448516296710的文件,我把cube
drop了重新构建了一个cube,还是会去找这个3271559448516296710文件,不知道哪里还存了这个元数据,
自己手动把对应的全局的GROUP_ID字典的索引目录都删了,重新创建一个cube
在去构建,还是报对应的错误。

是有什么地方需要清理缓存吗?


我现在把全局的这个字典删除了 变成了普通的字段,就可以构建成功,但是构建任务是按时间分区构建的,这样是否会影响
count_distinct的值的准确性。

Mime
View raw message