hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-550) Serialize ApplyTransaction calls per Container in ContainerStateMachine
Date Fri, 12 Oct 2018 00:50:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16647237#comment-16647237
] 

Hudson commented on HDDS-550:
-----------------------------

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15183 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15183/])
HDDS-550. Serialize ApplyTransaction calls per Container in (jitendra: rev 0bc6d0484a1c3652234b76e4f99e2ef99173796d)
* (edit) hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientRatis.java
* (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/XceiverServerGrpc.java
* (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/RDBStore.java
* (edit) hadoop-hdds/common/src/main/java/org/apache/ratis/RatisHelper.java
* (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/ContainerProtocolCalls.java
* (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/GrpcXceiverService.java
* (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/GrpcReplicationService.java
* (edit) hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ChunkInputStream.java
* (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestContainerServer.java
* (edit) hadoop-hdds/common/pom.xml
* (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestCloseContainerHandler.java
* (delete) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestContainerStateMachine.java
* (edit) hadoop-hdds/common/src/main/resources/ozone-default.xml
* (edit) hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestHddsDispatcher.java
* (edit) hadoop-hdds/common/src/main/proto/DatanodeContainerProtocol.proto
* (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/GrpcReplicationClient.java
* (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestCloseContainerHandlingByClient.java
* (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java
* (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/statemachine/background/BlockDeletingService.java
* (edit) hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ChunkOutputStream.java
* (edit) hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkDatanodeDispatcher.java
* (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ScmConfigKeys.java
* (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ContainerTestHelper.java
* (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java
* (edit) hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/ChunkGroupOutputStream.java
* (edit) hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientGrpc.java
* (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java
* (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/RocksDBStore.java
* (edit) hadoop-project/pom.xml
* (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/SmallFileUtils.java
* (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/ChunkUtils.java


> Serialize ApplyTransaction calls per Container in ContainerStateMachine
> -----------------------------------------------------------------------
>
>                 Key: HDDS-550
>                 URL: https://issues.apache.org/jira/browse/HDDS-550
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>            Reporter: Shashikant Banerjee
>            Assignee: Shashikant Banerjee
>            Priority: Major
>             Fix For: 0.3.0
>
>         Attachments: HDDS-550.001.patch, HDDS-550.002.patch, HDDS-550.003.patch, HDDS-550.004.patch,
HDDS-550.005.patch, HDDS-550.006.patch, HDDS-550.007.patch, HDDS-550.008.patch
>
>
> As part of handling Node failures in Ozone, the block commit need to happen in order
inside ContainerStateMachine per container. With RATIS-341, it is guaranteed that the 
applyTransaction calls for committing the write chunks will be initiated only when the WriteStateMachine
data for write Chunk operations finish. 
> This Jira is aimed at making all the applyTransaction operations inside ContainerStateMachine
serial per container with a single thread Executor per container handling all applyTransactions
calls.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message