hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mukul Kumar Singh (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-12767) Ozone: Handle larger file sizes for OzoneOutputStream & OzoneInputStream
Date Thu, 02 Nov 2017 19:41:02 GMT
Mukul Kumar Singh created HDFS-12767:
----------------------------------------

             Summary: Ozone: Handle larger file sizes for OzoneOutputStream & OzoneInputStream
                 Key: HDFS-12767
                 URL: https://issues.apache.org/jira/browse/HDFS-12767
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: ozone
    Affects Versions: HDFS-7240
            Reporter: Mukul Kumar Singh
            Assignee: Mukul Kumar Singh
            Priority: Major
             Fix For: HDFS-7240


This is based on [~stevel@apache.org]'s comments on HDFS-7240. This jira will add capabilities
to OzoneOutputStream and OzoneInputStream stream to handle larger file sizes. This jira will
also address other Streams related review comments.

OzoneOutputStream

    Implement StreamCapabilities and declare that hsync/hflush are not supported.
    Unless there is no limit on the size of a PUT request/multipart uploads are supported,
consider
    having the stream's write(int) method fail when the limit is reached.
    That way, things will at least fail fast.
    after close, set backupStream = null.
    flush() should be a no-op if called on a closed stream, so if (closed) return
    write() must fail if called on a closed stream,
    Again, OzoneException -> IOE translation which could/should be eliminated.

OzoneInputStream
1)    You have chosen an interesting solution to the "efficient seek" problem here: D/L the
entire file and then seek around. While this probably works for the first release, larger
files will have problems in both disk space and size of
2)    Again, OzoneException -> IOE translation which could/should be eliminated.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message