kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 赵天烁 <zhaotians...@meizu.com>
Subject 回复: 答复: Extract Fact Table Distinct Columns Error while don't have any distinct columns---kylin 2.0 beta
Date Thu, 09 Mar 2017 08:53:44 GMT
ok,is there any way to walk around that issue?

________________________________
赵天烁
Kevin Zhao
zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>

珠海市魅族科技有限公司
MEIZU Technology Co., Ltd.
广东省珠海市科技创新海岸魅族科技楼
MEIZU Tech Bldg., Technology & Innovation Coast
Zhuhai, 519085, Guangdong, China
meizu.com

发件人: Fan Xie<mailto:fan.xie@kyligence.io>
发送时间: 2017-03-09 16:51
收件人: user@kylin.apache.org<mailto:user@kylin.apache.org>
主题: 答复: Extract Fact Table Distinct Columns Error while don't have any distinct
columns---kylin 2.0 beta

The FactDistinctColumnsMapper is used by all kinds of dictionary encoding dimensions. I guest
that there may be some very long values in some of your dimension in your test  data and it's
size exceed 4096 bytes. And thus cause a buffer overflow exception.

________________________________
发件人: 赵天烁 <zhaotianshuo@meizu.com>
发送时间: 2017年3月9日 15:56:47
收件人: user
主题: Extract Fact Table Distinct Columns Error while don't have any distinct columns---kylin
2.0 beta

run into java.nio.BufferOverflowException while building a cube:

Error: java.nio.BufferOverflowException at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:183)
at java.nio.ByteBuffer.put(ByteBuffer.java:832) at org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper.doMap(FactDistinctColumnsMapper.java:157)
at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:48) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) at java.security.AccessController.doPrivileged(Native
Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

but I don't have any distinct column,the only meature I have are as follow:
[cid:_Foxmail.1@a035c257-7349-2008-c921-14b6ff365591]
a count and two raw measure.I'm curious about does raw measure use the same kind of mapper
as count distinct ?


________________________________
赵天烁
Kevin Zhao
zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>

珠海市魅族科技有限公司
MEIZU Technology Co., Ltd.
广东省珠海市科技创新海岸魅族科技楼
MEIZU Tech Bldg., Technology & Innovation Coast
Zhuhai, 519085, Guangdong, China
meizu.com
Mime
View raw message