hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6423) Writes should not block reads on blocking updates to memstores
Date Tue, 20 Nov 2012 18:28:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501354#comment-13501354
] 

stack commented on HBASE-6423:
------------------------------

Needs release note of new config.

You think Long.MAX_VALUE as default?  In trunk why not have it a reasonable value and in 0.94
have your MAX_VALUE?

Should default be socket timeout?

What is this comment about?

+    // block waiting for the lock for closing
     lock.writeLock().lock();

No busy waiting when closing?

I see the comment in a few places.

Removing getRecentFlushInfo is ok because HRegion is annotated private?

This is a little ugly:

+      int multiplier = Math.min(acquiredLocks.size(), maxBusyWaitMultiplier);
+      lock(this.updatesLock.readLock(), multiplier);

You pass in multiplier.  Why not pass in calculated time to busy wait?  Have a little function
that consults multiplier and
does Math.min, etc., and figures how long to wait?

+      int multiplier = Math.min(acquiredLocks.size(), maxBusyWaitMultiplier);

For example, put the above into the lock method?  Pass in size?

Fix your capialization of the 'N' in +   * @throws RegioNTooBusyException if failed to get
the lock in time

Can we backport?

So write locks are unbounded, its just the reads that we'll give up on.  How does this address
Karthik's original comment in the issue?
Reads fail faster releasing handlers so we can read/write against other memstores other than
the one that has a write lock on it (and is 
currently at its limit)?

Good patch Jimmy

                
> Writes should not block reads on blocking updates to memstores
> --------------------------------------------------------------
>
>                 Key: HBASE-6423
>                 URL: https://issues.apache.org/jira/browse/HBASE-6423
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Karthik Ranganathan
>            Assignee: Jimmy Xiang
>         Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch
>
>
> We have a big data use case where we turn off WAL and have a ton of reads and writes.
We found that:
> 1. flushing a memstore takes a while (GZIP compression)
> 2. incoming writes cause the new memstore to grow in an unbounded fashion
> 3. this triggers blocking memstore updates
> 4. in turn, this causes all the RPC handler threads to block on writes to that memstore
> 5. we are not able to read during this time as RPC handlers are blocked
> At a higher level, we should not hold up the RPC threads while blocking updates, and
we should build in some sort of rate control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message