hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
Date Fri, 20 Nov 2015 06:50:10 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015321#comment-15015321
] 

stack commented on HBASE-13082:
-------------------------------

bq. 	    // Start the CompactedHFileCleaner here

This is new?

Where we explain what it does?

This comment is no longer correct, right?

	    // notify scanners, close file readers, and recompute store size

Has to be public because its in the Interface? Does it have to be:

	  public Collection<StoreFile> getCompactedfiles() {


Might want to note that expectation is that access on methods like this one are single-threaded:
clearCompactedFiles

Do you have to stop the chore in the region or store close? Before you do your close?

Not your change but remove it since it obviously wrong now:     // 4. Compute new store size

Yeah, this needs better explaining especially if in an Interface

67	  /**
68	   * Get the compacted store files
69	   * @return the list of compacted files
70	   */
71	  Collection<StoreFile> getCompactedfiles();

You need these in the Interface?

475	  boolean isPrimaryReplicaStore();
476	
477	  /**
478	   * Closes and archives the compacted files under this store
479	   */
480	  void closeAndArchiveCompactedFiles() throws IOException;
481	
482	  /**
483	   * Close and archive the compacted files under this store
484	   * @param compactedStorefiles the list of compacted files
485	   */
486	  void closeAndArchiveCompactedFiles(List<StoreFile> compactedStorefiles) throws
IOException;

Do they have to be so specific? Can they be made more generic?

Yeah, this is hard... we have both nomenclatures going on ... compacted and

	    ACTIVE, DISCARDED;
157	  }


This is in the StoreFile.

It seems like the compacted or not belongs in StoreFileInfo rather than in StoreFile. Is this
fact persisted across open/close?

Maybe 'compactedAway'?

Got as far as StoreFileManager.



> Coarsen StoreScanner locks to RegionScanner
> -------------------------------------------
>
>                 Key: HBASE-13082
>                 URL: https://issues.apache.org/jira/browse/HBASE-13082
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>            Assignee: ramkrishna.s.vasudevan
>         Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt, 13082-v4.txt, 13082.txt,
13082.txt, HBASE-13082.pdf, HBASE-13082_1.pdf, HBASE-13082_12.patch, HBASE-13082_13.patch,
HBASE-13082_14.patch, HBASE-13082_1_WIP.patch, HBASE-13082_2.pdf, HBASE-13082_2_WIP.patch,
HBASE-13082_3.patch, HBASE-13082_4.patch, HBASE-13082_9.patch, HBASE-13082_9.patch, HBASE-13082_withoutpatch.jpg,
HBASE-13082_withpatch.jpg, LockVsSynchronized.java, gc.png, gc.png, gc.png, hits.png, next.png,
next.png
>
>
> Continuing where HBASE-10015 left of.
> We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock
already held by the RegionScanner.
> In tests this shows quite a scan improvement and reduced CPU (the fences make the cores
wait for memory fetches).
> There are some drawbacks too:
> * All calls to RegionScanner need to be remain synchronized
> * Implementors of coprocessors need to be diligent in following the locking contract.
For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation
(not picking on Phoenix, this one is my fault as I told them it's OK)
> * possible starving of flushes and compaction with heavy read load. RegionScanner operations
would keep getting the locks and the flushes/compactions would not be able finalize the set
of files.
> I'll have a patch soon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message