hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shashikant Banerjee (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-12794) Ozone: Parallelize ChunkOutputSream Writes to container
Date Wed, 10 Jan 2018 21:23:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Shashikant Banerjee updated HDFS-12794:
    Attachment: HDFS-12794-HDFS-7240.009.patch

Thanks [~xyao], for the review comments.

Ratis tests are failing because ratis version needs update addressed with HDFS-12986 which
has changes related XceiverClient-Ratis as well. 
for TestContainerPersistence tests , filed a new jira HDFS-13006 because the tests are failing
without the patch as well.

SCMCli tests and container state tests are failing and HDFS-12968 is filed for same. Few other
cases timed out while they pass in local box.

Few tests like TestKsmBlockVersioning#testReadLatestVersion were failing because of :
          .checkState(streamBufferSize > 0 && streamBufferSize > chunkSize);

in ChunkGroupOutputStream.java which are fixed by setting streamBufferSize in MB correctly.

patch v9 addresses rest of your review comments. Please have a look.

> Ozone: Parallelize ChunkOutputSream Writes to container
> -------------------------------------------------------
>                 Key: HDFS-12794
>                 URL: https://issues.apache.org/jira/browse/HDFS-12794
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Shashikant Banerjee
>            Assignee: Shashikant Banerjee
>             Fix For: HDFS-7240
>         Attachments: HDFS-12794-HDFS-7240.001.patch, HDFS-12794-HDFS-7240.002.patch,
HDFS-12794-HDFS-7240.003.patch, HDFS-12794-HDFS-7240.004.patch, HDFS-12794-HDFS-7240.005.patch,
HDFS-12794-HDFS-7240.006.patch, HDFS-12794-HDFS-7240.007.patch, HDFS-12794-HDFS-7240.008.patch,
> The chunkOutPutStream Write are sync in nature .Once one chunk of data gets written,
the next chunk write is blocked until the previous chunk is written to the container.
> The ChunkOutputWrite Stream writes should be made async and Close on the OutputStream
should ensure flushing of all dirty buffers to the container.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message