hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-2152) Data transfer buffer length in DFSOutputStream should not be user controlled.
Date Tue, 06 Nov 2007 00:21:50 GMT
Data transfer buffer length in DFSOutputStream should not be user controlled.
-----------------------------------------------------------------------------

                 Key: HADOOP-2152
                 URL: https://issues.apache.org/jira/browse/HADOOP-2152
             Project: Hadoop
          Issue Type: Bug
    Affects Versions: 0.14.0
            Reporter: Konstantin Shvachko
             Fix For: 0.16.0


DFSOutputStream has two distinct buffers. 
First one is the user data buffer, which collects data the user writes into stream, 
and flushes it into a temporary local file when the buffer is full.
Second is the data transfer buffer, which is used to transfer data from the client to a data-node.
It turns out to be that both buffers have the same length defined by the user when the file
is created.
I think the two buffer lengths should be completely independent.
The user data buffer can still be controlled by the user, but the data transfer
buffer should have independent length optimized for better transfer rate.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message