Return-Path: Delivered-To: apmail-incubator-hama-dev-archive@minotaur.apache.org Received: (qmail 31835 invoked from network); 16 Dec 2010 03:56:04 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 16 Dec 2010 03:56:04 -0000 Received: (qmail 8302 invoked by uid 500); 16 Dec 2010 03:56:04 -0000 Delivered-To: apmail-incubator-hama-dev-archive@incubator.apache.org Received: (qmail 8220 invoked by uid 500); 16 Dec 2010 03:56:01 -0000 Mailing-List: contact hama-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hama-dev@incubator.apache.org Delivered-To: mailing list hama-dev@incubator.apache.org Received: (qmail 8210 invoked by uid 99); 16 Dec 2010 03:56:00 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Dec 2010 03:56:00 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of clin4j@googlemail.com designates 209.85.216.175 as permitted sender) Received: from [209.85.216.175] (HELO mail-qy0-f175.google.com) (209.85.216.175) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Dec 2010 03:55:54 +0000 Received: by qyk8 with SMTP id 8so318786qyk.6 for ; Wed, 15 Dec 2010 19:55:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=UMLdDjJ1r/zN3qqoatP0uQFie6bGxtJwQLE7fcVNwio=; b=FQyMuXdsECaGC52p+dENKC35ddQ7xjlvSqXnXLDHuenmRdYPQY32l4T1FU/sNEqtUc ErDaPY9eYlVUutcik8fJy85drZ7SmVjLs1qJI6FTaEp8lb1rQHotsm1Tw+xd4I+RYAFH 5ioqiisqUAdbQfADBCXHazTuLtw3mlgzh2AQc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=Y7QcFkTMCBF0e6B8VhMNo4sPafg+wh3eMGyBjH3LWRdMWpHBJsIF+BqBwTnA8IczwL xS+gpN1Bf9gzeBYyVD4IXy9e3H1Pucjs1+C3kcHwnQRs+vJnFdZRDqz+1bOfdmx8ynZY JiHNghlPSZdOM+9XTBUd0gNcV5xInRD2AKHvA= MIME-Version: 1.0 Received: by 10.229.74.147 with SMTP id u19mr1237173qcj.72.1292471733159; Wed, 15 Dec 2010 19:55:33 -0800 (PST) Received: by 10.229.230.212 with HTTP; Wed, 15 Dec 2010 19:55:33 -0800 (PST) Date: Thu, 16 Dec 2010 11:55:33 +0800 Message-ID: Subject: Task distribution issue From: Lin Chia-Hung To: hama-dev@incubator.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org When working on BSPPeer fault tolerance (https://issues.apache.org/jira/browse/HAMA-199), I came across another issue - the task distribution currently employed in HAMA is done by GroomServer requesting tasks from BSPMaster, similar to one used in Hadoop mapreduce. This strategy has discussion at https://issues.apache.org/jira/browse/MAPREDUCE-278 regarding to e.g memory footprint, race condition, etc. Although this issue do not directly relate to the BSPPeer fault tolerance, the strategy (GroomServer requests tasks from BSPMaster) employed may have potential issue e.g. task can not be reschedule to the node expected. So I would like to know if there is any chance the HAMA task distribution mechanism may work toward this direction (proactive task assignment)? Thanks