zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Flavio Paiva Junqueira (JIRA)" <j...@apache.org>
Subject [jira] Commented: (ZOOKEEPER-486) Improve bookie performance for large number of ledgers
Date Wed, 02 Dec 2009 08:58:20 GMT

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784717#action_12784717

Flavio Paiva Junqueira commented on ZOOKEEPER-486:

Here is the log output for the failing test:

2009-12-02 09:46:22,952 - INFO  - [main:BookieClientTest@198] - Starting no ledger test
2009-12-02 09:46:22,956 - INFO  - [Thread-4:BookieClientTest$1@75] - Capacity 16, 0
2009-12-02 09:46:22,957 - INFO  - [Thread-4:BookieClientTest$1@83] - Received 0
2009-12-02 09:46:23,001 - ERROR - [SyncThread:Bookie$LastLogMark@295] - Problems writing to
java.io.FileNotFoundException: /var/folders/iq/iqmyL+XiGjqE2AjRkKTe-E+++TI/-Tmp-/bookie2550661167481866393test/lastMark
(No such file or directory)
	at java.io.FileOutputStream.open(Native Method)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
	at org.apache.bookkeeper.bookie.Bookie$LastLogMark.rollLog(Bookie.java:290)
	at org.apache.bookkeeper.bookie.Bookie$SyncThread.run(Bookie.java:116)
2009-12-02 09:46:23,002 - FATAL - [Thread-3:Bookie@375] - Bookie thread exiting
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1899)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1934)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
	at org.apache.bookkeeper.bookie.Bookie.run(Bookie.java:341)

and the assert statement that fails:

Testcase: testNoLedger took 0.132 sec
expected:<1> but was:<2>
junit.framework.AssertionFailedError: expected:<1> but was:<2>
	at org.apache.bookkeeper.test.BookieClientTest.testNoLedger(BookieClientTest.java:204)

It seems to me that the problem is that the bookie is not processing the FileNotFoundException
properly. It should send a ENOLEDGER value back. 

> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>                 Key: ZOOKEEPER-486
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
>             Project: Zookeeper
>          Issue Type: Improvement
>          Components: contrib-bookkeeper
>            Reporter: Flavio Paiva Junqueira
>            Assignee: Benjamin Reed
>         Attachments: ZOOKEEPER-486.patch
> If we write simultaneously to a large number of ledgers on a bookie, then performance
drops significantly due to more seeks on the ledger device. In such cases, we should be clustering
ledgers into files to reduce the number of seeks, and performing sequential writes on each
file. Clustering ledgers will impact read performance, so we would to have a knob to control
such a feature.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message