hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15016) StoreServices facility in Region
Date Wed, 30 Dec 2015 20:20:49 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075376#comment-15075376
] 

stack commented on HBASE-15016:
-------------------------------

Ok. I tried you patch. Here is some feedback.

So, RegionStoresProxy is meant to be a list of all the 'services' or functions a Store needs
of its encompassing Region. I'd think that a Region would Implement this Interface or maybe
if we want to keep this all contained inside this package, yeah, you might have a protected
method on Region that returns the provider.

So, I'd think that we'd want the Store to get all its Region services via this new facility
rather than just a few else we'll be in an intermediate place with half-done stuff. So, if
I go to HStore and comment out HRegion, I see that HRegion is needed to do the following:

{code}
  @Override
  public long getMemstoreFlushSize() {
    // TODO: Why is this in here?  The flushsize of the region rather than the store?  St.Ack
    return this.region.memstoreFlushSizeLB;
  }
...
To get the CP Host

    info.setRegionCoprocessorHost(this.region.getCoprocessorHost());
....

To get an executor pool....


        // initialize the thread pool for closing store files in parallel.
        ThreadPoolExecutor storeFileCloserThreadPool = this.region
            .getStoreFileOpenAndCloseThreadPool("StoreFileCloserThread-"
                + this.getColumnFamilyName());

.....

For some functionality that actually doesn't work....


    InetSocketAddress[] favoredNodes = null;
    if (region.getRegionServerServices() != null) {
      favoredNodes = region.getRegionServerServices().getFavoredNodesForRegion(
          region.getRegionInfo().getEncodedName());
    }


.....
To write a compaction marker to the WAL

    HRegionInfo info = this.region.getRegionInfo();
    CompactionDescriptor compactionDescriptor = ProtobufUtil.toCompactionDescriptor(info,
        family.getName(), inputPaths, outputPaths, fs.getStoreDir(getFamily().getNameAsString()));
    WALUtil.writeCompactionMarker(region.getWAL(), this.region.getTableDesc(),
        this.region.getRegionInfo(), compactionDescriptor, region.getMVCC());


.....

For some weird reporting reason....

    this.region.reportCompactionRequestStart(request.isMajor());


....

and here:     this.region.reportCompactionRequestEnd(cr.isMajor(), cr.getFiles().size(), cr.getSize());


....

and for these:



  @Override
  public RegionCoprocessorHost getCoprocessorHost() {
    return this.region.getCoprocessorHost();
  }

  @Override
  public HRegionInfo getRegionInfo() {
    return this.fs.getRegionInfo();
  }

  @Override
  public boolean areWritesEnabled() {
    return this.region.areWritesEnabled();
  }

  @Override
  public long getSmallestReadPoint() {
    return this.region.getSmallestReadPoint();
  }
{code}

You'd add blockUpdates/unblockUpdates which seems reasonable and getWalSequenceId, though
you are getting a new sequenceid.... which we should look at carefully...  and then for three
reads on memstore state that I think need better specification:

{code}
  public long addAndGetGlobalMemstoreSize(long size) {
    return this.region.addAndGetGlobalMemstoreSize(size);
  }

  public long addAndGetGlobalMemstoreFluctuatingSize(long size) {
    return this.memstoreFluctuatingSize.addAndGet(size);
  }

  public long getGlobalMemstoreActiveSize() {
{code}

I'm not clear on differences between global memstore size, fluctuating size, and active size.
 I think users will be confused.  Anyways, the Store being able to add/subtract to the Region-level
memory usage accounting is totally reasonable.

I have other review but maybe I should just press on and complete a version of this patch
of yours to illustrate better what I mean by the review? If that would help, just say and
I'll have a go at it.

> StoreServices facility in Region
> --------------------------------
>
>                 Key: HBASE-15016
>                 URL: https://issues.apache.org/jira/browse/HBASE-15016
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>         Attachments: HBASE-15016-V01.patch, HBASE-15016-V02.patch, HBASE-15016-V03.patch
>
>
> The default implementation of a memstore ensures that between two flushes the memstore
size increases monotonically. Supporting new memstores that store data in different formats
(specifically, compressed), or that allows to eliminate data redundancies in memory (e.g.,
via compaction), means that the size of the data stored in memory can decrease even between
two flushes. This requires memstores to have access to facilities that manipulate region counters
and synchronization.
> This subtasks introduces a new region interface -- StoreServices, through which store
components can access these facilities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message