hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HAMA-516) Implement Worker Pool
Date Fri, 24 Feb 2012 15:16:48 GMT
Implement Worker Pool

                 Key: HAMA-516
                 URL: https://issues.apache.org/jira/browse/HAMA-516
             Project: Hama
          Issue Type: Improvement
    Affects Versions: 0.4.0
            Reporter: Thomas Jungblut
            Priority: Minor
             Fix For: 0.5.0

To decrease job startup time we should implement worker pools.

Worker pools should start BSPTask JVM's based on the configured task capacity.
This should greatly improve cold-start time for jobs. However, this cost is quite low compared
to the long-running Hama task.

The idea is from http://www.slideshare.net/hanborq/hanborq-optimizations-on-hadoop-mapreduce-20120216a
(slide 4). Google Tenzing uses this, and I read about the gmail priority inbox jobs which
also uses this task reuse.

This will be the start of a number of tasks that will profile and improve startup time of
jobs and cluster. (Umbrella follows).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message