hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sri Ramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1788) Increase the buffer size of pipes from 1k to 128k
Date Mon, 15 Oct 2007 12:47:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534843
] 

Amareshwari Sri Ramadasu commented on HADOOP-1788:
--------------------------------------------------


Thanks Christian.

I ran my 500KB write example with this patch and now, there is 128KB write followed by the
rest.

mmap2(NULL, 507904, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7ec4000
write(3, "oooooooooooooooooooooooooooooooo"..., 131072) = 131072
write(3, "oooooooooooooooooooooooooooooooo"..., 393216) = 393216
munmap(0xb7ec4000, 507904)              = 0

Patch is fine, except some clarifications
1. 128*1028 has to be replaced by 128*1024.
2. do we need the change in procedure progress() ?
4. do we require setsockopt change now?
3. should we do setvbuf for the case of 'hadoop.pipes.command.file'  environment also?

> Increase the buffer size of pipes from 1k to 128k
> -------------------------------------------------
>
>                 Key: HADOOP-1788
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1788
>             Project: Hadoop
>          Issue Type: Bug
>          Components: pipes
>            Reporter: Owen O'Malley
>            Assignee: Amareshwari Sri Ramadasu
>         Attachments: patch-1788-1.txt, patch-1788.txt
>
>
> Currently pipes applications use 1k writes to the socket and it should be larger to increase
throughput.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message