hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-11135) Change region sequenceid generation so happens earlier in the append cycle rather than just before added to file
Date Sat, 10 May 2014 22:05:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994282#comment-13994282
] 

stack edited comment on HBASE-11135 at 5/10/14 7:03 PM:
--------------------------------------------------------

[~jeffreyz] has been reviewing offline.  Comments that if CP is slow doing pre/post append,
then as is, we will slow throughput.  But as he notes, this is as it is in 0.98 and not really
a way around it since cp, and wal listeners are inline w/ the actual append.

In this patch over most of the appendNoSyncs to use new method (so edit/sequence id is availble
on return out of append).  Changed the append to use a latch.  This gained me back a few percent...
maybe 5%... but this patch makes us slower appending.  When 200 threads contending, about
half slower again (180seconds vs 280 seconds to complete test):

{code}
$ for i in 1 3 5 10 50 200; do for j in 1 2 3; do perf stat ./hbase-0.99.0-SNAPSHOT/bin/hbase
--config /home/stack/conf_hbase org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation
-threads $i   -iterations 100000 -keySize 50 -valueSize 100  &> "/tmp/wpatch_latch_barrier.${i}.${j}.txt";
done; done
{code}

I tried then also using a cyclicbarrier in the SyncFuture for the handler thread and the the
sync'ing thread to coordinate around but no noticeable difference (perhaps a little slower).

I'd be good committing this for now since it simplifies the unification of mvcc and sequence
id.  Even when we are half slower again we are still about 0.98 throughput in trunk since
the claim in HBASE-10156 is that it doubled our throughput at 200 threads.

After the unification happens, we can come back here again to do see if we can get perf back.


was (Author: stack):
[~jeffreyz] has been reviewing offline.  Comments that if CP is slow doing pre/post append,
then as is, we will slow throughput.  But as he notes, this is as it is in 0.98 and not really
a way around it since cp, and wal listeners are inline w/ the actual append.

In this patch over most of the appendNoSyncs to use new method (so edit/sequence id is availble
on return out of append).  Changed the append to use a latch.  This gained me back a few percent...
maybe 5%... but this patch makes us slower appending.  When 200 threads contending, about
1/3rd slower again (180seconds vs 280 seconds to complete test):

{code}
$ for i in 1 3 5 10 50 200; do for j in 1 2 3; do perf stat ./hbase-0.99.0-SNAPSHOT/bin/hbase
--config /home/stack/conf_hbase org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation
-threads $i   -iterations 100000 -keySize 50 -valueSize 100  &> "/tmp/wpatch_latch_barrier.${i}.${j}.txt";
done; done
{code}

I tried then also using a cyclicbarrier in the SyncFuture for the handler thread and the the
sync'ing thread to coordinate around but no noticeable difference (perhaps a little slower).

I'd be good committing this for now since it simplifies the unification of mvcc and sequence
id.  Even when we are 1/3rd slower we are still ahead of 0.98 in trunk since the claim in
HBASE-10156 is that it doubled our throughput at 200 threads. 

After the unification happens, we can come back here again to do see if we can get perf back.

> Change region sequenceid generation so happens earlier in the append cycle rather than
just before added to file
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-11135
>                 URL: https://issues.apache.org/jira/browse/HBASE-11135
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>            Reporter: stack
>            Assignee: stack
>         Attachments: 11135.wip.txt, 11135v2.txt, 11135v5.txt
>
>
> Currently we assign the region edit/sequence id just before we put it in the WAL.  We
do it in the single thread that feeds from the ring buffer.  Doing it at this point, we can
ensure order, that the edits will be in the file in accordance w/ the ordering of the region
sequence id.
> But the point at which region sequence id is assigned an edit is deep down in the WAL
system and there is a lag between our putting an edit into the WAL system and the edit actually
getting its edit/sequence id.
> This lag -- "late-binding" -- complicates the unification of mvcc and region sequence
id, especially around async WAL writes (and, related, for no-WAL writes) -- the parent for
this issue (For async, how you get the edit id in our system when the threads have all gone
home -- unless you make them wait?)
> Chatting w/ Jeffrey Zhong yesterday, we came up with a crazypants means of getting the
region sequence id near-immediately.  We'll run two ringbuffers.  The first will mesh all
handler threads and the consumer will generate ids (we will have order on other side of this
first ring buffer), and then if async or no sync, we will just let the threads return ...
updating mvcc just before we let them go.  All other calls will go up on to the second ring
buffer to be serviced as now (batching, distribution out among the sync'ing threads).  The
first rb will have no friction and should turn at fast rates compared to the second.  There
should not be noticeable slowdown nor do I foresee this refactor intefering w/ our multi-WAL
plans.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message