hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-1702) Reduce buffer copies when data is written to DFS
Date Sat, 23 Feb 2008 01:23:22 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571673#action_12571673
] 

rangadi edited comment on HADOOP-1702 at 2/22/08 5:22 PM:
---------------------------------------------------------------

The attached patch reduces buffer copies on DataNode. DataNode read one 
packet and writes it to mirror and the local disk. Each packet contains
io.file.buffer.size of data. 

Below: 'b.i.s' is BufferedInputStream and 'b.o.s' is BufferedOutputStream, and these are
are much larger than typical size of read or write to them. 

Each {{--->}} represents a memory copy.

- Datanode : 
-- before :{noformat}
                                                     + ---> b.o.s ---> mirror socket
                                                     |
  client socket ---> b.i.s ---> small DataNode buf --|
                                                     |
                                                     + ---> b.o.s ---> local disk 
  {noformat}
-- after : {noformat}
                                                   + ---> mirror socket
                                                   |
  client socket ---> Large Datanode buf (Packet) --|
                                                   |
                                                   + ---> local disk 
  {noformat}
  
- Client : 
-- before: Client used 64k packets irrespective of io.file.buffer.size. 
   So the extra copy for b.o.s was not present if io.file.buffer.size at or
   below 64k. But each packet required 2 writes.
-- after : the size of packet is based on io.file.buffer.size and we use one
   write to write to datanode socket.    


I don't have numbers regd cpu savings. In absolute numbers, for given amount of data  on one
DataNode, CPU saved on DataNode should be larger than CPU saved when same amount is read with
HADOOP-2758.

DFSIO benchmark numbers have been very sensitive to buffering (not to CPU) while writing,
we need to show this patch does not negatively affect this benchmark.


      was (Author: rangadi):
    The attached patch reduces buffer copies on DataNode. DataNode read one 
packet and writes it to mirror and the local disk. Each packet contains
io.file.buffer.size of data.  This patch 


Below: 'b.i.s' is BufferedInputStream and 'b.o.s' is BufferedOutputStream, and these are
are much larger than typical size of read or write to them. 

- Datanode : 
-- before :{noformat}
                                                     + ---> b.o.s ---> mirror socket
                                                     |
  client socket ---> b.i.s ---> small DataNode buf --|
                                                     |
                                                     + ---> b.o.s ---> local disk 
  {noformat}
-- after : {noformat}
                                                   + ---> mirror socket
                                                   |
  client socket ---> Large Datanode buf (Packet) --|
                                                   |
                                                   + ---> local disk 
  {noformat}
  
- Client : 
-- before: Client used 64k packets irrespective of io.file.buffer.size. 
   So the extra copy for b.o.s was not present if io.file.buffer.size at or
   below 64k. But each packet required 2 writes.
-- after : the size of packet is based on io.file.buffer.size and we use one
   write to write to datanode socket.    


I don't have numbers regd cpu savings. In absolute numbers, for given amount of data  on one
DataNode, CPU saved on DataNode should be larger than CPU saved when same amount is read with
HADOOP-2758.

DFSIO benchmark numbers have been very sensitive to buffering (not to CPU) while writing,
we need to show this patch does not negatively affect this benchmark.

  
> Reduce buffer copies when data is written to DFS
> ------------------------------------------------
>
>                 Key: HADOOP-1702
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1702
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.14.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>             Fix For: 0.17.0
>
>         Attachments: HADOOP-1702.patch
>
>
> HADOOP-1649 adds extra buffering to improve write performance.  The following diagram
shows buffers as pointed by (numbers). Each eatra buffer adds an extra copy since most of
our read()/write()s match the io.bytes.per.checksum, which is much smaller than buffer size.
> {noformat}
>        (1)                 (2)          (3)                 (5)
>    +---||----[ CLIENT ]---||----<>-----||---[ DATANODE ]---||--<>-> to
Mirror  
>    | (buffer)                  (socket)           |  (4)
>    |                                              +--||--+
>  =====                                                    |
>  =====                                                  =====
>  (disk)                                                 =====
> {noformat}
> Currently loops that read and write block data, handle one checksum chunk at a time.
By reading multiple chunks at a time, we can remove buffers (1), (2), (3), and (5). 
> Similarly some copies can be reduced when clients read data from the DFS.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message