kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <roberto.tar...@stratebi.com>
Subject RE: Dict value is too long
Date Fri, 20 Apr 2018 10:07:39 GMT
Hi,

 

Looks like there is a dimension with high cardinality (HC) that does not fit
in dictionary. Please check if you have a HC dimension and try to change its
Rowkey Encoding from dictionary (dic) to other encoding, such as fixed
length. Have you tried this already?

 

Regards,

 

 

 

From: Lu Zhe [mailto:tcluzhe@hotmail.com] 
Sent: martes, 10 de abril de 2018 9:37
To: user@kylin.apache.org
Subject: 答复: Dict value is too long

 

抱歉,我修改下格式。

 

java.lang.IllegalStateException: maxValueLength is negative (-26578). Dict
value is too long, whose length is larger than 32767
 at org.apache.kylin.dict.TrieDictionary.init(TrieDictionary.java:104)
 at org.apache.kylin.dict.TrieDictionary.readFields(TrieDictionary.java:339)
 at
org.apache.kylin.dict.lookup.SnapshotTable.readData(SnapshotTable.java:250)
 at
org.apache.kylin.dict.lookup.SnapshotTableSerializer.deserialize(SnapshotTab
leSerializer.java:74)
 at
org.apache.kylin.dict.lookup.SnapshotTableSerializer.deserialize(SnapshotTab
leSerializer.java:48)
 at
org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.
java:171)
 at
org.apache.kylin.dict.lookup.SnapshotManager.load(SnapshotManager.java:196)
 at
org.apache.kylin.dict.lookup.SnapshotManager.checkDupByContent(SnapshotManag
er.java:178)
 at
org.apache.kylin.dict.lookup.SnapshotManager.trySaveNewSnapshot(SnapshotMana
ger.java:140)
 at
org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.j
ava:120)
 at
org.apache.kylin.cube.CubeManager$DictionaryAssist.buildSnapshotTable(CubeMa
nager.java:1055)
 at
org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:971)
 at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGe
neratorCLI.java:87)
 at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGe
neratorCLI.java:49)
 at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob
.java:71)
 at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:97)
 at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellEx
ecutable.java:63)
 at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable
.java:162)
 at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChaine
dExecutable.java:67)
 at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable
.java:162)
 at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultS
cheduler.java:300)
 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
42)
 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
17)
 at java.lang.Thread.run(Thread.java:745)

result code:2

 

原本怀疑是字段值过长,因为上次遇到的情况就是这样,后来我限制了字段值的长度就
好了。
这次又遇到这个问题,不是很理解。

 

  _____  

发件人: Lu Zhe < <mailto:tcluzhe@hotmail.com> tcluzhe@hotmail.com>
发送时间: 2018年4月10日 15:29
收件人:  <mailto:user@kylin.apache.org> user@kylin.apache.org
主题: Dict value is too long 

 

 Kylin 2.3.1





Build Dimension Dictionary时报错:

java.lang.IllegalStateException: maxValueLength is negative (-26578). Dict
value is too long, whose length is larger than 32767
                    at
org.apache.kylin.dict.TrieDictionary.init(TrieDictionary.java:104)
                    at
org.apache.kylin.dict.TrieDictionary.readFields(TrieDictionary.java:339)
                    at
org.apache.kylin.dict.lookup.SnapshotTable.readData(SnapshotTable.java:250)
                    at org.apache.kylin.dict.lookup.SnapshotTableSerializer.
deserialize(SnapshotTableSerializer.java:74)
                    at org.apache.kylin.dict.lookup.SnapshotTableSerializer.
deserialize(SnapshotTableSerializer.java:48)
                    at
org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.
java:171)
                    at
org.apache.kylin.dict.lookup.SnapshotManager.load(SnapshotManager.java:196)
                    at
org.apache.kylin.dict.lookup.SnapshotManager.checkDupByContent(SnapshotManag
er.java:178)
                    at
org.apache.kylin.dict.lookup.SnapshotManager.trySaveNewSnapshot(SnapshotMana
ger.java:140)
                    at
org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.j
ava:120)
                    at
org.apache.kylin.cube.CubeManager$DictionaryAssist.buildSnapshotTable(CubeMa
nager.java:1055)
                    at
org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:971)
                    at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGe
neratorCLI.java:87)
                    at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGe
neratorCLI.java:49)
                    at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.
run(CreateDictionaryJob.java:71)
                    at
org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:97)
                    at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellEx
ecutable.java:63)
                    at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable
.java:162)
                    at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChaine
dExecutable.java:67)
                    at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable
.java:162)
                    at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultS
cheduler.java:300)
                    at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
42)
                    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
17)
                    at java.lang.Thread.run(Thread.java:745)
 
result code:2

 

原本怀疑是字段值过长,因为上次遇到的情况就是这样,后来我限制了字段值的长度就
好了。

这次又遇到这个问题,不是很理解。


Mime
View raw message