hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-40) bufferSize argument is ignored in FileSystem.create(File, boolean, int)
Date Thu, 23 Feb 2006 17:23:38 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-40?page=all ]

Doug Cutting updated HADOOP-40:

    Attachment: bufferSize.patch

I don't think we should modify the configuration in this case, since that will affect code
which uses this configuration that runs later.  SequenceFile uses very large buffers when
sorting and merging, in order to minimize disk seeks, and we don't want everything to start
using such large buffers.

So why not just pass the missing parameter down?  I've attached a patch that does this.  Does
this look good to you?

> bufferSize argument is ignored in FileSystem.create(File, boolean, int)
> -----------------------------------------------------------------------
>          Key: HADOOP-40
>          URL: http://issues.apache.org/jira/browse/HADOOP-40
>      Project: Hadoop
>         Type: Bug
>   Components: fs
>     Reporter: Konstantin Shvachko
>     Priority: Minor
>  Attachments: BufferSize.patch
> org.apache.hadoop.fs.FileSystem.create(File f, boolean overwrite, int bufferSize)
> ignores the input parameter bufferSize.
> It passes further down the internal configuration, which includes the buffer size, but
not the parameter value.
> This works fine within the file system, since everything that calls create extracts buffer
size from the same config. 
> MapReduce although is probably affected by that, see 
> org.apache.hadoop.io.SequenceFile.Sorter.MergeQueue.MergeQueue(int size, String outName,
boolean done)
> The attached patch would fix it.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message