hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4841) If I call split fast enough, while inserting, rows disappear.
Date Wed, 30 Nov 2011 19:09:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160242#comment-13160242
] 

Jean-Daniel Cryans commented on HBASE-4841:
-------------------------------------------

Sorry if I'm not being clear.

So the claim is that this test shows we can lose rows while splitting too fast. I tried the
test myself on both 0.92 and trunk multiple times.

The only times I got errors (not failures, which would mean it's failing an assertion), is
when we call admin.split every so often in the test:

{code}
    for (int i = 0; i != NUMBER_OF_ROWS; i++) {
      byte[] rowName = Bytes.toBytes(i);
      Put put = new Put(rowName);
      for (int j = 0; j != NUMBER_OF_COLS; j++) {
        put.add(CF, String.valueOf(j).getBytes(), Bytes.toBytes(j * i));
      }

      if (i % NUM_ROWS_BEFORE_SPLIT == 0 && i != 0) {
        admin.split(TABLE_NAME);
        LOG.info("Splitting");
      }

      htable.put(put);

      if (i % NUM_ROWS_BEFORE_OUTPUT == 0 && i!= 0 ) {
        LOG.info("Inserted Row:" + i);
}
{code}

The problem is that if you call split on a table that has a region offline, the exception
bubbles all the way up and, in this case, kills the test. That's why I needed to catch and
move forward.

After this change, the test passes 100% of the time for both 0.92 and trunk.

Now what I'm wondering is if the test in Alex's case was failing or "erroring". If the former,
it's either an unknown bug or HBASE-4853. If the latter, then it's the issue I saw and there's
no data loss.
                
> If I call split fast enough, while inserting, rows disappear. 
> --------------------------------------------------------------
>
>                 Key: HBASE-4841
>                 URL: https://issues.apache.org/jira/browse/HBASE-4841
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Alex Newman
>            Assignee: ramkrishna.s.vasudevan
>            Priority: Critical
>         Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you
can get to the point to where the cluster becomes unstable, or rows will  disappear. The unit
test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split. 
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On
my macbook air, i could actually turn down the number of total rows, and the frequency of
the splits which is surprising. I think this is because the macbook air has much better IO
than my backup acer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message