hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15031) Fix merge of MVCC and SequenceID performance regression in branch-1.0
Date Mon, 28 Dec 2015 18:09:49 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-15031:
--------------------------
    Release Note: 
Increments can be 10x slower (or more) when there is high concurrency since HBase 1.0.0 (HBASE-8763).

This 'fix' adds back a fast increment but speed is achieved by relaxing row-level consistency
for Increments (only). The default remains the old, slow, consistent Increment behavior.

Set  "hbase.increment.fast.but.narrow.consistency" to true in hbase-site.xml to enable 'fast'
increments and then rolling restart your cluster. This is a setting the server-side needs
to read.

Intermixing fast increment with other Mutations will give indeterminate results; e.g. a Put
and Increment against the same Cell will not always give you the result you expect. Fast Increments
are consistent unto themselves. A Get with {@link IsolationLevel#READ_UNCOMMITTED} will return
the latest increment value or an Increment of an amount zero will do the same (beware doing
Get on a cell that has not been incremented yet -- this will return no results).

The difference between fastAndNarrowConsistencyIncrement and slowButConsistentIncrement is
that the former holds the row lock until the WAL sync completes; this allows us to reason
that there are no other writers afoot when we read the current increment value. In this case
we do not need to wait on mvcc reads to catch up to writes before we proceed with the read
of the current Increment value, the root of the slowdown seen in HBASE-14460. The fast-path
also does not wait on mvcc to complete before returning to the client (but the write has been
synced and put into memstore before we return). 

Also adds a simple performance test tool that will run against existing cluster. It expects
the table to be already created (by default it expects the table 'tableName' with a column
family 'columnFamilyName'): 

{code}
$ ./bin/hbase org.apache.hadoop.hbase.IncrementPerformanceTest
{code]

Configure it by passing -D options. Here are the set below:

2015-12-23 19:33:36,941 INFO  [main] hbase.IncrementPerformanceTest: Running test with hbase.zookeeper.quorum=localhost,
tableName=tableName, columnFamilyName=columnFamilyName, threadCount=80, incrementCount=10000

... so to set the tableName pass -DtableName=SOME_TABLENAME

  was:
Increments can be 10x slower (or more) when there is high concurrency since HBase 1.0.0 (HBASE-8763).

This 'fix' adds back a fast increment but speed is achieved by relaxing row-level consistency
for Increments (only). The default remains the old, slow, consistent Increment behavior.

Set  "hbase.increment.fast.but.narrow.consistency" to true in hbase-site.xml to enable 'fast'
increments and then rolling restart your cluster. This is a setting the server-side needs
to read.

Intermixing fast increment with other Mutations will give indeterminate results; e.g. a Put
and Increment against the same Cell will not always give you the result you expect. Fast Increments
are consistent unto themselves. A Get with {@link IsolationLevel#READ_UNCOMMITTED} will return
the latest increment value or an Increment of an amount zero will do the same (beware doing
Get on a cell that has not been incremented yet -- this will return no results).

The difference between fastAndNarrowConsistencyIncrement and slowButConsistentIncrement is
that the former holds the row lock until the WAL sync completes; this allows us to reason
that there are no other writers afoot when we read the current increment value. In this case
we do not need to wait on mvcc reads to catch up to writes before we proceed with the read
of the current Increment value, the root of the slowdown seen in HBASE-14460. The fast-path
also does not wait on mvcc to complete before returning to the client (but the write has been
synced and put into memstore before we return). 

Also adds a simple performance test tool that will run against existing cluster: 

{code}
$ ./bin/hbase org.apache.hadoop.hbase.IncrementPerformanceTest
{code]

Configure it by passing -D options. Here are the set below:

2015-12-23 19:33:36,941 INFO  [main] hbase.IncrementPerformanceTest: Running test with hbase.zookeeper.quorum=localhost,
tableName=tableName, columnFamilyName=[B@610ac287, threadCount=80, incrementCount=10000

... so to set the tableName pass -DtableName=SOME_TABLENAME


> Fix merge of MVCC and SequenceID performance regression in branch-1.0
> ---------------------------------------------------------------------
>
>                 Key: HBASE-15031
>                 URL: https://issues.apache.org/jira/browse/HBASE-15031
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Performance
>    Affects Versions: 1.0.3
>            Reporter: stack
>            Assignee: stack
>             Fix For: 1.0.4
>
>         Attachments: 14460.v0.branch-1.0.patch, 15031.v2.branch-1.0.patch, 15031.v3.branch-1.0.patch,
15031.v4.branch-1.0.patch, 15031.v5.branch-1.0.patch, 15031.v6.branch-1.0.patch, 15031.v6.branch-1.0.patch,
15031.v6.branch-1.0.patch
>
>
> Subtask with fix for branch-1.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message