hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Liochon (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
Date Wed, 20 Nov 2013 18:26:36 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827933#comment-13827933
] 

Nicolas Liochon edited comment on HBASE-10001 at 11/20/13 6:25 PM:
-------------------------------------------------------------------

There is a bug linked to this:
in HRegion#doMiniBatchMutation:
we start with:
{code}boolean walSyncSuccessful = false;{code}

if the preBatch hook call byPass, we then return immediately, because of this:
{code}if (coprocessorHost.preBatchMutate(miniBatchOp)) return 0L;{code}

then the finally clause is executed with walSyncSuccessful set to false, and then we try to
rollback the memstore.

{code}
      if (!walSyncSuccessful) {
        rollbackMemstore(batchOp, familyMaps, firstIndex, lastIndexExclusive);
      }
{code}

In my load test, we're spending 15% of our time in the rollback.

I can fix this in the coprocessor (by not setting bypass), but I think we should set the value
of walSyncSuccessful later, just before we try to write something in the memstore:


Any opinion? I can do the fix in HRegion#doMiniBatchMutation or in the coprocessor.




was (Author: nkeywal):
There is a bug linked to this:
in HRegion#doMiniBatchMutation:
we start with:
??boolean walSyncSuccessful = false;??

if the preBatch hook call byPass, we then return immediately, because of this:
?? if (coprocessorHost.preBatchMutate(miniBatchOp)) return 0L;??

then the finally clause is executed with ??walSyncSuccessful?? set to false, and then we try
to rollback the memstore.
In my load test, we're spending 15% of our time in the rollback.

I can fix this in the coprocessor (by not setting bypass), but I think we should set the value
of walSyncSuccessful later, just before we try to write something in the memstore.

Any opinion? I can do the fix in HRegion#doMiniBatchMutation or in the coprocessor.



> Add a coprocessor to help testing the performances without taking into account the i/o
> --------------------------------------------------------------------------------------
>
>                 Key: HBASE-10001
>                 URL: https://issues.apache.org/jira/browse/HBASE-10001
>             Project: HBase
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 0.98.0, 0.96.0, 0.94.13
>            Reporter: Nicolas Liochon
>            Assignee: Nicolas Liochon
>            Priority: Minor
>             Fix For: 0.98.0, 0.96.1, 0.94.15
>
>         Attachments: 10001.v1.patch, 10001.v2.patch
>
>
> We have a mockup to test only the client. If we want to include the network, without
beeing limited by the i/o, we don't have much tools.
> This coprocessor helps to test this.
> I put it in the main code as to make it usable without adding a jar...
> I don't think it's possible avoid the WAL writes in the coprocessors. It would be great
to have it to simplify the test with any kind of client (i.e. w/o changing the durability).



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message