hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-1381) The distance between sync blocks in SequenceFiles should be configurable
Date Fri, 25 Nov 2016 16:55:58 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Harsh J updated HADOOP-1381:
       Resolution: Fixed
     Hadoop Flags: Reviewed
    Fix Version/s: 3.0.0-beta1
     Release Note: The default sync interval within new SequenceFile writes is now 100KB,
up from the older default of 2000B. The sync interval is now also manually configurable via
the SequenceFile.Writer API.  (was: Made sync interval of sequencefiles configurable and raised
default from 2000 bytes to 100 kilobytes, to optimize for large files.)
           Status: Resolved  (was: Patch Available)

Thank you [~ajisakaa]! Pushed to trunk.

> The distance between sync blocks in SequenceFiles should be configurable
> ------------------------------------------------------------------------
>                 Key: HADOOP-1381
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1381
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io
>    Affects Versions: 2.0.0-alpha
>            Reporter: Owen O'Malley
>            Assignee: Harsh J
>             Fix For: 3.0.0-beta1
>         Attachments: HADOOP-1381.r1.diff, HADOOP-1381.r2.diff, HADOOP-1381.r3.diff, HADOOP-1381.r4.diff,
HADOOP-1381.r5.diff, HADOOP-1381.r5.diff
> Currently SequenceFiles put in sync blocks every 2000 bytes. It would be much better
if it was configurable with a much higher default (1mb or so?).

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message