hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gregory Chanan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-6950) TestAcidGuarantees system test now flushes too aggressively
Date Thu, 04 Oct 2012 21:23:47 GMT
Gregory Chanan created HBASE-6950:
-------------------------------------

             Summary: TestAcidGuarantees system test now flushes too aggressively
                 Key: HBASE-6950
                 URL: https://issues.apache.org/jira/browse/HBASE-6950
             Project: HBase
          Issue Type: Bug
          Components: test
    Affects Versions: 0.92.2, 0.94.2, 0.96.0
            Reporter: Gregory Chanan
            Assignee: Gregory Chanan
            Priority: Minor
             Fix For: 0.96.0


HBASE-6552 caused the TestAcidGuarantees system test to flush more aggressively, because flushes
are where ACID problems have occurred in the past.

After some more cluster testing, it seems like this too aggressive; my clusters eventually
can't keep up with the number of flushes/compactions and start getting SocketTimeoutExceptions.
 We could try to optimize the flushes/compactions, but since this workload would never occur
in practice, I don't think it is worth the effort.  Instead, let's just only flush once a
minute.  This is arbitrary, but seems to work.

Here is my comment in the (upcoming) patch:
{code}
// Flushing has been a source of ACID violations previously (see HBASE-2856), so ideally,
// we would flush as often as possible.  On a running cluster, this isn't practical:
// (1) we will cause a lot of load due to all the flushing and compacting
// (2) we cannot change the flushing/compacting related Configuration options to try to
// alleviate this
// (3) it is an unrealistic workload, since no one would actually flush that often.
// Therefore, let's flush every minute to have more flushes than usual, but not overload
// the running cluster.
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message