kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "lk_hadoop"<lk_had...@163.com>
Subject can't pass step Build Cube In-Mem
Date Tue, 09 Apr 2019 01:21:59 GMT
hi,all :
   I'm using kylin-2.6.1-cdh57, and the source row count is 500 million´╝îI can success build
cube . 
   but when I use the cube planner , it has one step : Build Cube In-Mem for job :OPTIMIZE
CUBE
   the config about the kylin_job_conf_inmem.xml is :

   <property>
        <name>mapreduce.map.memory.mb</name>
        <value>9216</value>
        <description></description>
    </property>

    <property>
        <name>mapreduce.map.java.opts</name>
        <value>-Xmx8192m -XX:OnOutOfMemoryError='kill -9 %p'</value>
        <description></description>
    </property>

    <property>
        <name>mapreduce.job.is-mem-hungry</name>
        <value>true</value>
    </property>
 
    <property>
        <name>mapreduce.job.split.metainfo.maxsize</name>
        <value>-1</value>
        <description>The maximum permissible size of the split metainfo file.
            The JobTracker won't attempt to read split metainfo files bigger than
            the configured value. No limits if set to -1.
        </description>
    </property>

    <property>
        <name>mapreduce.job.max.split.locations</name>
        <value>2000</value>
        <description>No description</description>
    </property>

    <property>
        <name>mapreduce.task.io.sort.mb</name>
        <value>200</value>
        <description></description>
    </property>


    finally the map job will be killed for OnOutOfMemoryError  , but when I giev more mem
for map job , I will get another error :  java.nio.BufferOverflowException
      
    why kylin will run the job inmem ? how can I avoid it ?
       


2019-04-08


lk_hadoop 
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message