hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-10351) Ozone: Optimize key writes to chunks by providing a bulk write implementation in ChunkOutputStream.
Date Fri, 29 Apr 2016 23:44:12 GMT

     [ https://issues.apache.org/jira/browse/HDFS-10351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Nauroth updated HDFS-10351:
    Attachment: HDFS-10351-HDFS-7240.001.patch

Quoting the JavaDocs of the bulk {{OutputStream#write}} method:

The write method of OutputStream calls the write method of one argument on each of the bytes
to be written out. Subclasses are encouraged to override this method and provide a more efficient

Well, they were sure right!  Profiling revealed a ton of function call overhead from sitting
in the base class {{OutputStream#write}}, iterating over the buffer, and calling the single-byte
write method repeatedly.  The attached patch provides a bulk write implementation in the subclass.
 I have measured a ~30% clock time performance improvement in putting a key.  Profiling after
the change shows that this isn't a hotspot anymore.

There are no new tests in this patch.  I'm relying on the existing tests or regression testing,
and I'm relying on the performance measure/profiling I described above as proof that the optimization

> Ozone: Optimize key writes to chunks by providing a bulk write implementation in ChunkOutputStream.
> ---------------------------------------------------------------------------------------------------
>                 Key: HDFS-10351
>                 URL: https://issues.apache.org/jira/browse/HDFS-10351
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HDFS-10351-HDFS-7240.001.patch
> HDFS-10268 introduced the {{ChunkOutputStream}} class as part of end-to-end integration
of Ozone receiving key content and writing it to chunks in a container.  That patch provided
an implementation of the mandatory single-byte write method.  We can improve performance by
adding an implementation of the bulk write method too.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message