hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elek, Marton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-325) Add event watcher for delete blocks command
Date Mon, 13 Aug 2018 16:48:01 GMT

    [ https://issues.apache.org/jira/browse/HDDS-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16578607#comment-16578607

Elek, Marton commented on HDDS-325:

Thank you the clarification [~ljain]. Let me summarize the goals (and fix me if my understanding
is wrong):

1. You would like to use two different event_type/topics. If the event is sent to the SCMEvents.DATANODE_COMMAND,
the message will be sent to the datanode and the retry logic should be implemented manually
and an additional message should be sent to an event watcher.

2. If the message is sent to the SCMEvents.RETRIABLE_DATANODE_COMMAND the message will be
sent to the datanode AND the retry will be handled automatically.

If this is the situation: I like the idea but I don't think that we need to implement a RetriableCompletionPayload
for that.  I think it could be implemented with a more simple way.
As I wrote I prefer to keep the messaging logic out from the message payload. Message payload
is just a collection of the data and I can't see any reason to put more logic to there. Even
in your patch you can put the retry logic to RetriableEventWatcher.onTimeout method.

The method payload type and the event type is separated. It's very easy to create a RetriableEventWatcher
which listens on the SCMEvents.RETRIABLE_DATANODE_COMMAND but can receive exactly the same
message as the SCMEvents.DATANODE_COMMAND. (It also could help to switch between the two approach).

I think currently we use a generic SCMEvents.DATANODE_COMMAND for all the datanode commands.
So we don't need to make the EventWatcher more complex with adding a Set to it. If in the
future we will switch to handle all the different datanode commands with different event type
we can easily instantiate multiple RetriableEventWatcher. With this approach we can monitor
the different type of events easily and the implementation could be more easy (IMHO).

Does it make sense?

> Add event watcher for delete blocks command
> -------------------------------------------
>                 Key: HDDS-325
>                 URL: https://issues.apache.org/jira/browse/HDDS-325
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>          Components: Ozone Datanode, SCM
>            Reporter: Lokesh Jain
>            Assignee: Lokesh Jain
>            Priority: Major
>             Fix For: 0.2.1
>         Attachments: HDDS-325.001.patch, HDDS-325.002.patch, HDDS-325.003.patch
> This Jira aims to add watcher for deleteBlocks command. It removes the current rpc call
required for datanode to send the acknowledgement for deleteBlocks.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message