hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3550) Reduce tasks failing with OOM
Date Fri, 13 Jun 2008 08:02:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12604767#action_12604767
] 

Arun C Murthy commented on HADOOP-3550:
---------------------------------------

Ok, hprof squeaked more:

{noformat}
SITES BEGIN (ordered by live bytes) Fri Jun 13 07:51:54 2008
          percent          live          alloc'ed  stack class
 rank   self  accum     bytes objs     bytes  objs trace name
    1 49.28% 49.28% 127078480 25039 475983328 6696233 376386 byte[]
    2 44.80% 94.08% 115532056    1 171655272     2 377442 byte[]
    3  3.30% 97.38%   8511552 25061 1513552256 4881857 376504 byte[]
    4  0.71% 98.09%   1835232   14   1835232    14 372662 byte[]
    5  0.71% 98.80%   1835232   14   1835232    14 372912 byte[]
    6  0.23% 99.03%    599184    9 174961728  2628 376627 byte[]
    7  0.16% 99.19%    400960 25060  78111920 4881857 376505 byte[]
    8  0.05% 99.24%    131088    1    131088     1 334784 byte[]
    9  0.05% 99.29%    131088    1    131088     1 375533 byte[]
   10  0.05% 99.34%    131088    1    131088     1 376424 byte[]
   11  0.05% 99.39%    131088    1    262176     2 377077 byte[]
   12  0.03% 99.42%     65552    1     65552     1 309878 byte[]
SITES END
{noformat}

Here are the traces for the top two:

{noformat}
TRACE 376386: (thread=200001)
  org.apache.hadoop.io.SequenceFile$UncompressedBytes.reset(SequenceFile.java:604)
  org.apache.hadoop.io.SequenceFile$UncompressedBytes.access$900(SequenceFile.java:594)
  org.apache.hadoop.io.SequenceFile$Reader.nextRawValue(SequenceFile.java:2022)
  org.apache.hadoop.io.SequenceFile$Sorter$SegmentDescriptor.nextRawValue(SequenceFile.java:3014)
  org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.next(SequenceFile.java:2758)
  org.apache.hadoop.mapred.ReduceTask$ValuesIterator.readNextKey(ReduceTask.java:272)
  org.apache.hadoop.mapred.ReduceTask$ValuesIterator.next(ReduceTask.java:233)
  org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:311)
  org.apache.hadoop.streaming.PipeReducer.reduce(PipeReducer.java:67)
  org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
  org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
{noformat}

and

{noformat}
TRACE 377442: (thread=200001)
  java.util.Arrays.copyOf(Arrays.java:2786)
  java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
  java.io.DataOutputStream.write(DataOutputStream.java:90)
  org.apache.hadoop.io.SequenceFile$UncompressedBytes.writeUncompressedBytes(SequenceFile.java:617)
  org.apache.hadoop.mapred.ReduceTask$ValuesIterator.readNextValue(ReduceTask.java:289)
  org.apache.hadoop.mapred.ReduceTask$ValuesIterator.next(ReduceTask.java:232)
  org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:311)
  org.apache.hadoop.streaming.PipeReducer.reduce(PipeReducer.java:67)
  org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
  org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
{noformat}

> Reduce tasks failing with OOM
> -----------------------------
>
>                 Key: HADOOP-3550
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3550
>             Project: Hadoop Core
>          Issue Type: Bug
>    Affects Versions: 0.17.0
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>            Priority: Blocker
>             Fix For: 0.17.1, 0.18.0
>
>
> Map-Reduce jobs which worked with 0.16.* are failing with OOM with the following trace:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
>         at java.util.Arrays.copyOf(Arrays.java:2786)
>         at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>         at java.io.DataOutputStream.write(DataOutputStream.java:90)
>         at org.apache.hadoop.io.SequenceFile$UncompressedBytes.writeUncompressedBytes(SequenceFile.java:617)
>         at org.apache.hadoop.mapred.ReduceTask$ValuesIterator.readNextValue(ReduceTask.java:289)
>         at org.apache.hadoop.mapred.ReduceTask$ValuesIterator.next(ReduceTask.java:232)
>         at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:311)
>         at org.apache.hadoop.streaming.PipeReducer.reduce(PipeReducer.java:67)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
>         at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message