hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Reed (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2315) BookKeeper for write-ahead logging
Date Fri, 12 Mar 2010 19:18:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12844628#action_12844628
] 

Benjamin Reed commented on HBASE-2315:
--------------------------------------

one thing that would be great for BookKeeper is to try to characterize your load and then
run some saturation tests to see how the performance looks. previously we varied messages
sizes from 128 bytes to 8K. should we try something like 100 bytes, 1000 bytes, 10K, 100K,
1M?

we should also vary the number of clients writing. what is a good number? 10-1000 or 10-10000?
is the WAL per server or per region. 

do you have WAL throughput numbers for the throughput you are getting from HDFS?

just to give you an idea, we are in the 10s of thousands of messages/sec for 1K messages.
(the performance increases as you add more servers, aka bookies.) for larger messages it really
is the network bandwidth that becomes the bottleneck.

> BookKeeper for write-ahead logging
> ----------------------------------
>
>                 Key: HBASE-2315
>                 URL: https://issues.apache.org/jira/browse/HBASE-2315
>             Project: Hadoop HBase
>          Issue Type: New Feature
>          Components: regionserver
>            Reporter: Flavio Paiva Junqueira
>         Attachments: HBASE-2315.patch, zookeeper-dev-bookkeeper.jar
>
>
> BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high throughput
write-ahead logging service. This issue provides an implementation of write-ahead logging
for hbase using BookKeeper. Apart from expected throughput improvements, BookKeeper also has
stronger durability guarantees compared to the implementation currently used by hbase.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message