Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
Date: Thu, 23 Jan 2014 10:01:44 +0000 (UTC)
From: "anselme dewavrin (JIRA)" <jira@apache.org>
To: dev@activemq.apache.org
Message-ID: <JIRA.12690266.1390397406949.193.1390471304199@arcas>
In-Reply-To: <JIRA.12690266.1390397406949@arcas>
References: <JIRA.12690266.1390397406949@arcas>
Subject: [jira] [Updated] (AMQ-4987) io wait on replicated levelDB slaves
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/AMQ-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

anselme dewavrin updated AMQ-4987:
----------------------------------

    Description: 
Dear all,
I set up a 3-nodes replicatedLevelDB activeMQ cluster on 3 different machines, as explained on the activemq site (with zookeeper etc.).

I made a message injector using the php stomp client described here : http://stomp.fusesource.org/documentation/php/book.html

Then I injected persistent messages as fast as possible (giving about 100 messages/s, each message is 10k).

Everything works fine, then I measured the servers' activity with "vmstat 1". I saw no iowait on the master node, but  20% on both slaves. This would impeach scalabitity I suppose. And the iowait is justified by 3000 bo/s (blocks out) in the vmstat report.

The machines are not swapping (paging).

Here is what I tried, without success :
-specify explicitly sync="quorum_mem"
-JNI implementation of the leveldb store (and verified it is used)
-setting flushDelay to 2000

Does anyone have an idea that I could try ? Why is the leveldb slaves writing so much to disk ?

Many thanks in advance
Yours,
Anselme

  was:
Dear all,
I set up a 3-nodes replicatedLevelDB activeMQ cluster as explained on the activemq site.

I made a message injector using the php stomp client described here : http://stomp.fusesource.org/documentation/php/book.html

Then I injected persistent messages as fast as possible (giving about 100 messages/s, each message is 10k).

Everything works fine, then I measured the servers' activity with "vmstat 1". I saw no iowait on the master node, but  20% on both slaves. This would impeach scalabitity I suppose. And the iowait is justified by 3000 bo/s (blocks out) in the vmstat report.

The machines are not swapping (paging).

Here is what I tried, without success :
-specify explicitly sync="quorum_mem"
-JNI implementation of the leveldb store (and verified it is used)
-setting flushDelay to 2000

Does anyone have an idea that I could try ? Why is the leveldb slaves writing so much to disk ?

Many thanks in advance
Yours,
Anselme


> io wait on replicated levelDB slaves
> ------------------------------------
>
>                 Key: AMQ-4987
>                 URL: https://issues.apache.org/jira/browse/AMQ-4987
>             Project: ActiveMQ
>          Issue Type: Test
>          Components: activemq-leveldb-store
>    Affects Versions: 5.9.0
>         Environment: debian VM 2.6.32-5-amd64, jdk7
>            Reporter: anselme dewavrin
>            Priority: Minor
>
> Dear all,
> I set up a 3-nodes replicatedLevelDB activeMQ cluster on 3 different machines, as explained on the activemq site (with zookeeper etc.).
> I made a message injector using the php stomp client described here : http://stomp.fusesource.org/documentation/php/book.html
> Then I injected persistent messages as fast as possible (giving about 100 messages/s, each message is 10k).
> Everything works fine, then I measured the servers' activity with "vmstat 1". I saw no iowait on the master node, but  20% on both slaves. This would impeach scalabitity I suppose. And the iowait is justified by 3000 bo/s (blocks out) in the vmstat report.
> The machines are not swapping (paging).
> Here is what I tried, without success :
> -specify explicitly sync="quorum_mem"
> -JNI implementation of the leveldb store (and verified it is used)
> -setting flushDelay to 2000
> Does anyone have an idea that I could try ? Why is the leveldb slaves writing so much to disk ?
> Many thanks in advance
> Yours,
> Anselme


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)