Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 4 Nov 2015 17:00:29 +0000 (UTC)
From: "Ted Yu (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12903188.1444261633000.152798.1446656429877@Atlassian.JIRA>
In-Reply-To: <JIRA.12903188.1444261633000@Atlassian.JIRA>
References: <JIRA.12903188.1444261633000@Atlassian.JIRA>
 <JIRA.12903188.1444261633077@arcas>
Subject: [jira] [Commented] (HBASE-14575) Reduce scope of compactions
 holding region lock
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14989905#comment-14989905 ] 

Ted Yu commented on HBASE-14575:
--------------------------------

We are trying to remove / relax the region read lock for compaction.
Let's see what are the potential race conditions among the operations (user scan, region split, region close and region bulk load).

user scan  ---> region read lock
region split --> region close first --> region write lock
region close  --> region write lock
region bulk load  --> region write lock

read lock is compatible with read lock.  ---> no problem with user scan/read
region bulk load does not cause problem for compaction (no consistency problem, store lock will help the store file accounting).
They can run almost concurrently at the region level.

The only remaining race condition is between the region close and compaction.
So we will evaluate, below, how region close intervenes with compaction if compaction does not acquire region read lock.

Here are the steps for compaction:
1. obtain list of StoreFile's

2. create StoreFileScanner's based on list from #1

3. perform compaction and save resulting files under tmp dir

4. swap in compacted files

#1 is guarded by store lock.  This patch does not change this --> no worse or better

For #2, we obtain smallest read point (for region) across all the Scanners (for both default compactor and stripe compactor).
The read points are for user scans. Region keeps the read points for all currently open user scanners.
Compaction needs to know the smallest read point so that during re-write of the hfiles, it can remove the mvcc points for the cells if their mvccs are older than the smallest since they are not needed anymore.
This will not conflict with compaction.

For #3, it can be performed in parallel to other operations.
#4 needs region read lock - this requirement is kept in patch.

Region close and compaction are guarded pretty well by the 'writestate'.
In HRegion#doClose(), we have :
{code}
    synchronized (writestate) {
      // Disable compacting and flushing by background threads for this
      // region.
      canFlush = !writestate.readOnly;
      writestate.writesEnabled = false;
      LOG.debug("Closing " + this + ": disabling compactions & flushes");
      waitForFlushesAndCompactions();
    }
{code}
waitForFlushesAndCompactions() would wait for writestate.compacting to come down to 0.
 
and in HRegion.compact()
{code}
      try {
        synchronized (writestate) {
          if (writestate.writesEnabled) {
            wasStateSet = true;
            ++writestate.compacting;
          } else {
            String msg = "NOT compacting region " + this + ". Writes disabled.";
            LOG.info(msg);
            status.abort(msg);
            return false;
          }
        }
{code}
Also in compactor.performCompaction():
{code}
       // check periodically to see if a system stop is requested
          if (closeCheckInterval > 0) {
            bytesWritten += len;
            if (bytesWritten > closeCheckInterval) {
              bytesWritten = 0;
              if (!store.areWritesEnabled()) {
                progress.cancel();
                return false;
              }
            }
          }
{code}

> Reduce scope of compactions holding region lock
> -----------------------------------------------
>
>                 Key: HBASE-14575
>                 URL: https://issues.apache.org/jira/browse/HBASE-14575
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Compaction, regionserver
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>         Attachments: 14575-v1.patch, 14575-v2.patch, 14575-v3.patch, 14575-v4.patch, 14575-v5.patch, 14575.v00.patch
>
>
> Per [~devaraj]'s idea on parent issue, let's see if we can reduce the scope of critical section under which compactions hold the region read lock.
> Here is summary from parent issue:
> Another idea is we can reduce the scope of when the read lock is held during compaction. In theory the compactor only needs a region read lock while deciding what files to compact and at the time of committing the compaction. We're protected from the case of region close events because compactions are checking (every 10k bytes written) if the store has been closed in order to abort in such a case.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)