hivemall-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Makoto Yui (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (HIVEMALL-73) Reduce memory usage of each_top_k
Date Mon, 20 Feb 2017 07:53:44 GMT

     [ https://issues.apache.org/jira/browse/HIVEMALL-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Makoto Yui closed HIVEMALL-73.
------------------------------
    Resolution: Won't Fix

Won't fix for now.

> Reduce memory usage of each_top_k
> ---------------------------------
>
>                 Key: HIVEMALL-73
>                 URL: https://issues.apache.org/jira/browse/HIVEMALL-73
>             Project: Hivemall
>          Issue Type: Improvement
>            Reporter: Makoto Yui
>            Assignee: Makoto Yui
>            Priority: Minor
>
> Better to reduce memory usage in drainQueue().
> {code}
> 2017-02-16 05:56:22,378 FATAL [Thread-4] org.apache.hadoop.mapred.YarnChild: Error running
child : java.lang.OutOfMemoryError: Java heap space
>     at hivemall.tools.EachTopKUDTF.drainQueue(EachTopKUDTF.java:182)
>     at hivemall.tools.EachTopKUDTF.close(EachTopKUDTF.java:215)
>     at org.apache.hadoop.hive.ql.exec.UDTFOperator.closeOp(UDTFOperator.java:143)
>     at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:577)
>     at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588)
>     at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588)
>     at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:318)
>     at org.apache.hadoop.io.IOUtils.cleanup(IOUtils.java:237)
>     at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:459)
>     at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message