hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liyin Tang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-6968) Several HBase write perf improvement
Date Wed, 10 Oct 2012 00:10:02 GMT

     [ https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Liyin Tang updated HBASE-6968:
------------------------------

    Description: 
Here are 2 hbase write performance improvements recently:

1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when creating a HBaseConfiguraiton
object, it would parse the xml configuration files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
  newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
  newWriter = createWriter(fs, newPath, conf);
{code}


2) Change 2 hotspot synchronized functions into double locking pattern. So it shall remove
the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
  public synchronized void inc(String name, int amt) {	
    MetricsTimeVaryingRate m = get(name);	
    if (m == null) {	
      m = create(name);	
    }	
    m.inc(amt);	
  }
{code}

new:
{code}
  public void inc(String name, int amt) {	
    MetricsTimeVaryingRate m = get(name);	
    if (m == null) {	
      synchronized (this) {	
        if ((m = get(name)) == null) {	
          m = create(name);	
        }	
      }	
    }	
    m.inc(amt);	
  }
{code}
=====================
orig:
{code:title=MemStoreFlusher.java}
  public synchronized void reclaimMemStoreMemory() {	
    if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {	
      flushSomeRegions();	
    }
  }	
{code}
new:
{code}
  public void reclaimMemStoreMemory() {	
    if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {	
      flushSomeRegions();	
    }
  }	
  private synchronized void flushSomeRegions() {	
    if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) {	
      return; // double check the global memstore size inside of the synchronized block.	
    }	
 ...   
 }
{code}



  was:
Here are 2 hbase write performance improvements recently found out. 

1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when creating a HBaseConfiguraiton
object, it would parse the xml configuration files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
  newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
  newWriter = createWriter(fs, newPath, conf);
{code}


2) Change 2 hotspot synchronized functions into double locking pattern. So it shall remove
the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
  public synchronized void inc(String name, int amt) {	
    MetricsTimeVaryingRate m = get(name);	
    if (m == null) {	
      m = create(name);	
    }	
    m.inc(amt);	
  }
{code}

new:
{code}
  public void inc(String name, int amt) {	
    MetricsTimeVaryingRate m = get(name);	
    if (m == null) {	
      synchronized (this) {	
        if ((m = get(name)) == null) {	
          m = create(name);	
        }	
      }	
    }	
    m.inc(amt);	
  }
{code}
=====================
orig:
{code:title=MemStoreFlusher.java}
  public synchronized void reclaimMemStoreMemory() {	
    if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {	
      flushSomeRegions();	
    }
  }	
{code}
new:
{code}
  public void reclaimMemStoreMemory() {	
    if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {	
      flushSomeRegions();	
    }
  }	
  private synchronized void flushSomeRegions() {	
    if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) {	
      return; // double check the global memstore size inside of the synchronized block.	
    }	
 ...   
 }
{code}



    
> Several HBase write perf improvement
> ------------------------------------
>
>                 Key: HBASE-6968
>                 URL: https://issues.apache.org/jira/browse/HBASE-6968
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Liyin Tang
>
> Here are 2 hbase write performance improvements recently:
> 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when creating a
HBaseConfiguraiton object, it would parse the xml configuration files from disk, which is
not cheap operation.
> In HLog.java:
> orig:
> {code:title=HLog.java}
>   newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
> {code}
> new:
> {code}
>   newWriter = createWriter(fs, newPath, conf);
> {code}
> 2) Change 2 hotspot synchronized functions into double locking pattern. So it shall remove
the synchronization overhead in the normal case.
> orig:
> {code:title=HBaseRpcMetrics.java}
>   public synchronized void inc(String name, int amt) {	
>     MetricsTimeVaryingRate m = get(name);	
>     if (m == null) {	
>       m = create(name);	
>     }	
>     m.inc(amt);	
>   }
> {code}
> new:
> {code}
>   public void inc(String name, int amt) {	
>     MetricsTimeVaryingRate m = get(name);	
>     if (m == null) {	
>       synchronized (this) {	
>         if ((m = get(name)) == null) {	
>           m = create(name);	
>         }	
>       }	
>     }	
>     m.inc(amt);	
>   }
> {code}
> =====================
> orig:
> {code:title=MemStoreFlusher.java}
>   public synchronized void reclaimMemStoreMemory() {	
>     if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {	
>       flushSomeRegions();	
>     }
>   }	
> {code}
> new:
> {code}
>   public void reclaimMemStoreMemory() {	
>     if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) {	
>       flushSomeRegions();	
>     }
>   }	
>   private synchronized void flushSomeRegions() {	
>     if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) {	
>       return; // double check the global memstore size inside of the synchronized block.

>     }	
>  ...   
>  }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message