hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nandakumar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12519) Ozone: Add a Lease Manager to SCM
Date Sun, 08 Oct 2017 13:12:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16196104#comment-16196104
] 

Nandakumar commented on HDFS-12519:
-----------------------------------

Thanks [~linyiqun] for the feedback.
bq.This comment seems not very clear. LOG.isDebug() is not always needed when calling LOG.debug.
If there are some parameters passed, the check LOG.isDebug() is not needed.
The reason for having {{LOG.isDebugEnabled()}} is to avoid too many String operations when
debug is not enabled and debug message has parameters.
Consider the following cases:
Case 1:
{{LOG.debug("Starting LeaseManager#LeaseMonitor Thread")}}
Here we actually don't need {{LOG.isDebugEnabled()}}, it will add additional  overhead.
Case 2:
{{LOG.debug("string" + obj + "string" + obj + "string" + obj)}}
In this case there will be {{toString}} calls on the objects used in message and string concatenation
also happens, which might affect the performance. And these string operations are done even
if log level is not set to debug. Here it makes sense to do {{LOG.isDebugEnabled()}} check.

Performing {{LOG.isDebugEnabled()}} check always will not affect the performance that much,
and in future if someone does change the log message and make is something like the one in
case 2, it will help us.

Anyways, I will remove  {{LOG.isDebugEnabled()}} check wherever we are doing constant string
logging. Is this ok?

bq. I am thinking if the method public synchronized Lease<T> acquire(T resource, long
timeout) is needed. Different timeout value resource can just use separated lease manager
to deal with. Multiple timeout required resources make single lease manager inconveniently
to deal with.

I agree on this point, but even if we remove the timeout option while acquiring lease we will
be stuck with the same problem.
Lets assume we are running lease monitor in fixed interval of time
{noformat}
t1              t2             t3            t4             t5             t6            
t7
 |--------------|--------------|--------------|--------------|--------------|--------------|
        |                |       |       |      |        |
        A L1           R L1     A L2    A L3  R L2      R L3
{noformat}
A L - Acquire Lease
R L - Release Lease

We need to run monitor thread once the lease times out which will not always fall on our fixed
time interval. In the above scenario we will be releasing lease L1 during t3, but the lease
has expired long before, and will be releasing L2 and L3 at t5 which is not correct.

Sorry if I have confused you more, please let me know if it's not clear.

We can remove the finally part in {{LeaseMonitor#run}} as it's already running inside while
loop.
{code}
finally {
          run();
}
{code}

I will update the patch.


> Ozone: Add a Lease Manager to SCM
> ---------------------------------
>
>                 Key: HDFS-12519
>                 URL: https://issues.apache.org/jira/browse/HDFS-12519
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Anu Engineer
>            Assignee: Nandakumar
>              Labels: OzonePostMerge
>         Attachments: HDFS-12519-HDFS-7240.000.patch, HDFS-12519-HDFS-7240.001.patch
>
>
> Many objects, including Containers and pipelines can time out during creating process.
We need a way to track these timeouts. This lease Manager allows SCM to hold a lease on these
objects and helps SCM timeout waiting for creating of these objects.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message