Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 67793 invoked from network); 18 Dec 2007 04:21:05 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 18 Dec 2007 04:21:05 -0000 Received: (qmail 44530 invoked by uid 500); 18 Dec 2007 04:20:53 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 44505 invoked by uid 500); 18 Dec 2007 04:20:53 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 44496 invoked by uid 99); 18 Dec 2007 04:20:53 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Dec 2007 20:20:53 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO brutus.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Dec 2007 04:20:40 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id E29D771428B for ; Mon, 17 Dec 2007 20:20:43 -0800 (PST) Message-ID: <29699732.1197951643925.JavaMail.jira@brutus> Date: Mon, 17 Dec 2007 20:20:43 -0800 (PST) From: "Amar Kamat (JIRA)" To: hadoop-dev@lucene.apache.org Subject: [jira] Commented: (HADOOP-2419) HADOOP-1965 breaks nutch In-Reply-To: <31604166.1197575443207.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12552621 ] Amar Kamat commented on HADOOP-2419: ------------------------------------ The guess is that {{MapRunnableTest.java{}} assumes that {{MapTask.collect()}} is thread-safe. Which the earlier patch did not provide. So the change makes the call thread-safe. > HADOOP-1965 breaks nutch > ------------------------ > > Key: HADOOP-2419 > URL: https://issues.apache.org/jira/browse/HADOOP-2419 > Project: Hadoop > Issue Type: Bug > Components: mapred > Affects Versions: 0.16.0 > Reporter: Paul Saab > Assignee: Amar Kamat > Attachments: jobtasks.jsp.html, MapRunnableTest.java > > > When running nutch on trunk, nutch is unable to complete a fetch and the following exceptions are raised: > java.io.EOFException > at java.io.DataInputStream.readFully(DataInputStream.java:180) > at org.apache.nutch.protocol.Content.readFields(Content.java:158) > at org.apache.nutch.util.GenericWritableConfigurable.readFields(GenericWritableConfigurable.java:38) > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.spill(MapTask.java:536) > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(MapTask.java:474) > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$100(MapTask.java:248) > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$1.run(MapTask.java:413) > Exception in thread "SortSpillThread" java.lang.NegativeArraySizeException > at org.apache.hadoop.io.Text.readString(Text.java:388) > at org.apache.nutch.metadata.Metadata.readFields(Metadata.java:243) > at org.apache.nutch.protocol.Content.readFields(Content.java:151) > at org.apache.nutch.util.GenericWritableConfigurable.readFields(GenericWritableConfigurable.java:38) > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.spill(MapTask.java:536) > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(MapTask.java:474) > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$100(MapTask.java:248) > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$1.run(MapTask.java:413) > After reverting HADOOP-1965 nutch works just fine. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.