hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-3104) MultithreadMapRunner keeps consuming records even if trheads are not available
Date Thu, 27 Mar 2008 07:27:24 GMT
MultithreadMapRunner keeps consuming records even if trheads are not available
------------------------------------------------------------------------------

                 Key: HADOOP-3104
                 URL: https://issues.apache.org/jira/browse/HADOOP-3104
             Project: Hadoop Core
          Issue Type: Bug
    Affects Versions: 0.16.1
         Environment: all
            Reporter: Alejandro Abdelnur
            Assignee: Alejandro Abdelnur
            Priority: Critical
             Fix For: 0.16.2


The ExecutorService execute() method does not block when there are not threads available,
it queues up the runnables until there are threads. 

The problem is that all key/values are read and kept in memory for the task, with large datasets
this will create a OOM exception.

Have to figure out how to use the execute in blocking fashion.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message