hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-8747) Log when blocked longer than expected on a lock/synchronized block
Date Sat, 15 Jun 2013 20:47:19 GMT
stack created HBASE-8747:

             Summary: Log when blocked longer than expected on a lock/synchronized block
                 Key: HBASE-8747
                 URL: https://issues.apache.org/jira/browse/HBASE-8747
             Project: HBase
          Issue Type: Improvement
          Components: Usability
    Affects Versions: 0.95.2
            Reporter: stack
            Assignee: stack
            Priority: Critical

Recently, on an internal support thread, the support folks asked that HBase surface when it
is blocked up internally or an internal resource is highly contended.  I replied in essence
that this would be 'too hard'.  Our Todd instead suggested:

...straw man proposal:

- swap out all our use of sychronized (...) for ReentrantLock (probably most of our locking
is already based on j.u.c locks anyway for rwlock purposes)
- build a wrapper around ReentrantLock that looks something like:

class AwesomeLock implements Lock {
  AwesomeLock(Lock realLock, int expectedAcquisitionTime, String metricName) {
    this.realLock = realLock;
  public void lock() {
    if (realLock.tryLock(expectedAcquisitionTime, MILLIS)) { return; }
    Stopwatch sw = new Stopwatch();
    // collect realLock.getOwner()
    long elapsed = sw.elapsed() + expectedAcquisitionTime;
    if (elapsed > expected * 2) { LOG.warn("Waited too long for lock which was held by
owner " + owner); }
    collect metrics about how long the lock took, the lock name etc

Basically the idea is that any possible point of contention should emit log messages and metrics
when contended.

Let me do the above over the next week or so.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message