Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 40398 invoked from network); 27 Jun 2006 21:14:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 27 Jun 2006 21:14:07 -0000 Received: (qmail 79569 invoked by uid 500); 27 Jun 2006 21:14:06 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 79546 invoked by uid 500); 27 Jun 2006 21:14:06 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 79536 invoked by uid 99); 27 Jun 2006 21:14:05 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Jun 2006 14:14:05 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [209.237.227.198] (HELO brutus.apache.org) (209.237.227.198) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Jun 2006 14:14:05 -0700 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id C39B47141F5 for ; Tue, 27 Jun 2006 21:12:30 +0000 (GMT) Message-ID: <11767902.1151442750797.JavaMail.jira@brutus> Date: Tue, 27 Jun 2006 21:12:30 +0000 (GMT+00:00) From: "Milind Bhandarkar (JIRA)" To: hadoop-dev@lucene.apache.org Subject: [jira] Commented: (HADOOP-318) Progress in writing a DFS file does not count towards Job progress and can make the task timeout In-Reply-To: <3721347.1151007329825.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/HADOOP-318?page=comments#action_12418126 ] Milind Bhandarkar commented on HADOOP-318: ------------------------------------------ I have attached the patch for nutch to NUTCH-312. Affter the recent commmits, there are conflicts (espcecially in datanodeInfo etc). I am resolving those issues, and will provide a new patch for this issue soon. In the meanwhile, I am deleting the currrently attached patches. > Progress in writing a DFS file does not count towards Job progress and can make the task timeout > ------------------------------------------------------------------------------------------------ > > Key: HADOOP-318 > URL: http://issues.apache.org/jira/browse/HADOOP-318 > Project: Hadoop > Type: Bug > Components: mapred > Versions: 0.3.2 > Environment: all, but especially on big busy clusters > Reporter: Milind Bhandarkar > Assignee: Milind Bhandarkar > Fix For: 0.4.0 > > When a task writes to DFS file, depending on how busy the cluster is, it can timeout after 10 minutes by default, because the progress towards writing a DFS file does not count as progress of the task. The solution (patch is forthcoming) is to provide a way to callback reporter to report task progress from DFSOutputStream. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira