hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HADOOP-18) Crash with multiple temp directories
Date Mon, 06 Mar 2006 22:32:03 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-18?page=all ]
     
Doug Cutting resolved HADOOP-18:
--------------------------------

    Resolution: Invalid

Resoving at Rod's request.

> Crash with multiple temp directories
> ------------------------------------
>
>          Key: HADOOP-18
>          URL: http://issues.apache.org/jira/browse/HADOOP-18
>      Project: Hadoop
>         Type: Bug
>   Components: mapred
>     Reporter: Rod Taylor
>     Priority: Critical

>
> A brief read of the code indicated it may be possible to use multiple local directories
using something like the below:
>   <property>
>     <name>mapred.local.dir</name>
>     <value>/local,/local1,/local2</value>
>     <description>The local directory where MapReduce stores intermediate
>     data files.
>     </description>
>   </property>
> This failed with the below exception during either the generate or update phase (not
entirely sure which).
> java.lang.ArrayIndexOutOfBoundsException
>         at java.util.zip.CRC32.update(CRC32.java:51)
>         at org.apache.nutch.fs.NFSDataInputStream$Checker.read(NFSDataInputStream.java:92)
>         at org.apache.nutch.fs.NFSDataInputStream$PositionCache.read(NFSDataInputStream.java:156)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>         at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
>         at java.io.DataInputStream.readFully(DataInputStream.java:176)
>         at org.apache.nutch.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:55)
>         at org.apache.nutch.io.DataOutputBuffer.write(DataOutputBuffer.java:89)
>         at org.apache.nutch.io.SequenceFile$Reader.next(SequenceFile.java:378)
>         at org.apache.nutch.io.SequenceFile$Reader.next(SequenceFile.java:301)
>         at org.apache.nutch.io.SequenceFile$Reader.next(SequenceFile.java:323)
>         at org.apache.nutch.mapred.SequenceFileRecordReader.next(SequenceFileRecordReader.java:60)
>         at org.apache.nutch.segment.SegmentReader$InputFormat$1.next(SegmentReader.java:80)
>         at org.apache.nutch.mapred.MapTask$2.next(MapTask.java:106)
>         at org.apache.nutch.mapred.MapRunner.run(MapRunner.java:48)
>         at org.apache.nutch.mapred.MapTask.run(MapTask.java:116)
>         at org.apache.nutch.mapred.TaskTracker$Child.main(TaskTracker.java:604)

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message