kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 赵希欣 <zxxb...@163.com>
Subject Configuration Overwrites the container memory for MR task not work
Date Thu, 19 Jul 2018 05:47:24 GMT

Hi Kylin users/developers 
Kylin 2.3.0, hadoop 2.6.5 cdh 5.12 
A error occurred when I built a cube at step 2 Redistribute Flat Hive Table. I checked the
MR log and the log shows that reducers containers is running beyond physical memory limits

below is detail logs: 
Diagnostic Messages for this Task: 
Container [pid=50920,containerID=container_1518617687620_384770_01_000287] is running beyond
physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 2.6 GB of 2.1
GB virtual memory used. Killing container. 

then I try to overwrites the related config 
kylin.engine.mr.config-override.mapreduce.reduce.memory.mb = 4096 
kylin.engine.mr.config-override.mapreduce.reduce.java.opts = -Xmx3g 



then built the cube again, the the same error came again. 
I check the yarn config mapreduce.reduce.memory.mb = 2G. this time I'm fully confused. where
kylin get the 1GB physical memory config limit? why the overwrites not work? 



--

BR
Xixin
Mime
View raw message