hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-834) Datanode goes OOM based because of segment size
Date Wed, 14 Nov 2018 11:19:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16686381#comment-16686381

Hudson commented on HDDS-834:

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15425 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15425/])
HDDS-834. Datanode goes OOM based because of segment size. Contributed (shashikant: rev a94828170684793b80efdd76dc8a3167e324c0ea)
* (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java
* (edit) hadoop-hdds/common/src/main/resources/ozone-default.xml
* (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ScmConfigKeys.java

> Datanode goes OOM based because of segment size
> -----------------------------------------------
>                 Key: HDDS-834
>                 URL: https://issues.apache.org/jira/browse/HDDS-834
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>          Components: Ozone Datanode
>    Affects Versions: 0.3.0
>            Reporter: Mukul Kumar Singh
>            Assignee: Mukul Kumar Singh
>            Priority: Major
>             Fix For: 0.3.0, 0.4.0
>         Attachments: HDDS-834-ozone-0.3.001.patch, HDDS-834.001.patch
> Currently ratis segment size is set to 1GB. After RATIS-253, the entry size for a write
Chunk is not  counted towards the entry being written to Raft Log.
> This jira controls the segment size to 16KB which makes sure that the number of entries
with WriteChunk is limited to 64. This means with 16MB chunk, the total data pending in the
segment is 1GB.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message