hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jothi Padmanabhan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4396) sort on 400 nodes is now slower than in 18
Date Thu, 16 Oct 2008 16:17:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jothi Padmanabhan updated HADOOP-4396:
--------------------------------------

    Attachment: 4396-v3.patch

This is the patch I used for testing.
No, we do not notice stragglers either in 18 or when using this patch.
And you are right, the final writes and reads that reach the disk are governed by the io.file.buffer.size.
That is exactly why I did not try this change earlier

> sort on 400 nodes is now slower than in 18
> ------------------------------------------
>
>                 Key: HADOOP-4396
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4396
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Jothi Padmanabhan
>            Assignee: Jothi Padmanabhan
>            Priority: Blocker
>             Fix For: 0.19.0
>
>         Attachments: 4396-v3.patch
>
>
> Sort on 400 nodes on  hadoop release 18 takes about 29 minutes, but with the 19 branch
takes about 32 minutes. This behavior is consistent.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message