Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 3 May 2017 17:54:04 +0000 (UTC)
From: "Weiwei Yang (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13068481.1493742804000.117796.1493834044876@Atlassian.JIRA>
In-Reply-To: <JIRA.13068481.1493742804000@Atlassian.JIRA>
References: <JIRA.13068481.1493742804000@Atlassian.JIRA> <JIRA.13068481.1493742804266@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-11740) Ozone: Differentiate time interval
 for different DatanodeStateMachine state tasks
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Wed, 03 May 2017 17:54:10 -0000


    [ https://issues.apache.org/jira/browse/HDFS-11740?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1599=
5325#comment-15995325 ]=20

Weiwei Yang commented on HDFS-11740:
------------------------------------

Hi [~anu]

Thanks for your quick response, appreciate. I read your comments, I have so=
me thoughts.

bq. The original 90 seconds is the time that datanode would use to read the=
 various containers and make sure that it is ready to communicate with both=
 SCM and the world.

What I saw the time was spent on state transitions, given the heartbeat tim=
e is 30s (default), it moves "slowly" from one state to another which is no=
t supposed to behave like this.=20

bq. Right now, we have some missing pieces, we need to launch a background =
thread =E2=80=93 that does a directory scan for containers and volumes =E2=
=80=93 so that when SCM asks for container reports we are ready.

Yes this is probably still a missing piece, but it doesn't seem to relate t=
o this work. If dn is not ready for container reports, it would just respon=
se an error when SCM pulls, so SCM can try again later.=20

bq. So I am doubtful if we will gain anything by accelerating the initial b=
oot time.

Say dn needs time to scan containers/volumes and it takes a while, I am gue=
ssing that will be another (new) state. When we add that, it will be much e=
asier to utilize the work in this jira so that we can control the time nece=
ssary for this task, other than a hard coded heartbeat interval. This time =
could be very different on different data size, e.g few seconds on brand ne=
w cluster, minutes on large cluster (I am just saying).

I uploaded a patch, it doesn't have UT and I will add later if the patch is=
 on the right direction. Some note about the fix, datanode state machine cu=
rrently uses {{ScmConfigKeys#OZONE_SCM_HEARTBEAT_INTERVAL_SECONDS}} as the =
fixed state interval, it now looks like [^statemachine_1.png]; I am proposi=
ng to change to [^statemachine_2.png]. I added a {{EndpointTask}} interface=
, all end point tasks need to implement {{long getTaskDuration()}} to decid=
e how long to wait before scheduling the task. For {{VersionEndpointTask}} =
and {{RegisterEndpointTask}}, they simply return 0 which means no lag and d=
irectly schedule; for {{HeartbeatEndpointTask}}, it returns the heartbeat i=
nterval so it sends heartbeat with respect to that interval.

Hope it helps. Thanks

> Ozone: Differentiate time interval for different DatanodeStateMachine sta=
te tasks
> -------------------------------------------------------------------------=
--------
>
>                 Key: HDFS-11740
>                 URL: https://issues.apache.org/jira/browse/HDFS-11740
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>         Attachments: HDFS-11725-HDFS-7240.001.patch, statemachine_1.png, =
statemachine_2.png
>
>
> Currently datanode state machine transitioned between tasks in a fixed ti=
me interval, defined by {{ScmConfigKeys#OZONE_SCM_HEARTBEAT_INTERVAL_SECOND=
S}}, the default value is 30s. Once datanode is started, it will need 90s b=
efore transited to {{Heartbeat}} state, such a long lag is not necessary. P=
ropose to improve the logic of time interval handling, it seems only the he=
artbeat task needs to be scheduled in {{OZONE_SCM_HEARTBEAT_INTERVAL_SECOND=
S}} interval, rest should be done without any lagging.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org