kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ji, Hao Wei Jeffery" <Jeffery...@dnvgl.com>
Subject dimension table limit 300MB?
Date Sat, 07 Nov 2015 09:00:04 GMT
Hi,

I meet a problem in building the cube. Seems there's a limitation for size of dimension tables,
is there any way to tune the threshold?


java.lang.IllegalStateException: Table snapshot should be no greater than 300 MB, but TableDesc
[database=DEFAULT name=DIMCUSTOMERCURRENT] size is 506677558
                    at org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:95)
                    at org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:205)
                    at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:60)
                    at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41)
                    at org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:52)
                    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
                    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
                    at org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:62)
                    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
                    at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)
                    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
                    at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)
                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
                    at java.lang.Thread.run(Thread.java:745)

result code:2




**************************************************************************************
This e-mail and any attachments thereto may contain confidential information and/or information
protected by intellectual property rights for the exclusive attention of the intended addressees
named above. If you have received this transmission in error, please immediately notify the
sender by return e-mail and delete this message and its attachments. Unauthorized use, copying
or further full or partial distribution of this e-mail or its contents is prohibited.
**************************************************************************************

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message