ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "GAO Rui (JIRA)" <>
Subject [jira] [Commented] (AMBARI-11383) Enable async audit log and unfair lock by default in HDP
Date Wed, 27 Jul 2016 06:47:20 GMT


GAO Rui commented on AMBARI-11383:

Thanks for your contribution, [~sumitmohanty] .

Could you share your opinion about why make default value of dfs.namenode.fslock.fair as false?

In current HDFS source code, the default value is set as true: 
    boolean fair = conf.getBoolean("dfs.namenode.fslock.fair", true);"fsLock is fair:" + fair);
    fsLock = new FSNamesystemLock(fair);

And, according to the following document, for a busy cluster(where both write and read operations
concurrency are very high), it's very easy to cause the problem of writers can never get the
lock. Because, once one reader acquire the read lock and mounts of reader keep acquiring read
lock, the writers wound got the chance to acquire write lock at all.
        Non-fair mode (default)
        When constructed as non-fair (the default), the order of entry to the read and write
lock is unspecified, subject to reentrancy constraints. A nonfair lock that is continuously
contended may indefinitely postpone one or more reader or writer threads, but will normally
have higher throughput than a fair lock.
        Fair mode
        When constructed as fair, threads contend for entry using an approximately arrival-order
policy. When the currently held lock is released either the longest-waiting single writer
thread will be assigned the write lock, or if there is a group of reader threads waiting longer
than all waiting writer threads, that group will be assigned the read lock.
        A thread that tries to acquire a fair read lock (non-reentrantly) will block if either
the write lock is held, or there is a waiting writer thread. The thread will not acquire the
read lock until after the oldest currently waiting writer thread has acquired and released
the write lock. Of course, if a waiting writer abandons its wait, leaving one or more reader
threads as the longest waiters in the queue with the write lock free, then those readers will
be assigned the read lock.
        A thread that tries to acquire a fair write lock (non-reentrantly) will block unless
both the read lock and write lock are free (which implies there are no waiting threads). (Note
that the non-blocking ReentrantReadWriteLock.ReadLock.tryLock() and ReentrantReadWriteLock.WriteLock.tryLock()
methods do not honor this fair setting and will acquire the lock if it is possible, regardless
of waiting threads.)

> Enable async audit log and unfair lock by default in HDP
> --------------------------------------------------------
>                 Key: AMBARI-11383
>                 URL:
>             Project: Ambari
>          Issue Type: Bug
>          Components: stacks
>    Affects Versions: 2.1.0
>            Reporter: Sumit Mohanty
>            Assignee: Sumit Mohanty
>            Priority: Critical
>             Fix For: 2.1.0
>         Attachments: AMBARI-11383.patch
> Set the corresponding conf properties:
> dfs.namenode.audit.log.async=true 
> dfs.namenode.fslock.fair=false

This message was sent by Atlassian JIRA

View raw message